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PREFACE 


This  study  was  conducted  in  support  of  project  1121,  Technical  Training 
Development,  Dr.  Marty  R.  Rockway,  project  scientist;  task  1121-02,  Dr.  Joseph  Y. 
Yasutake,  task  scientist;  work  unit  1121-02-06,  PLATO  IV  Service  Test,  Brian  E. 
Dallman,  principal  investigator.  This  report  covers  research  conducted  between  August 
1972  and  June  1976. 

This  report  is  the  culmination  of  a 4-year  effort  to  examine  PLATO  IV’s 
assimilation  into  an  Air  Force  technical  training  environment.  Over  the  years  many 
individuals  performed  at  their  limit  under  difficult  conditions.  It  is  these  people  who 
make  a project  a success,  and  this  report  is  dedicated  to  them.  The  authors  would  like  to 
acknowledge  everyone  associated  with  the  project,  however,  it  is  practical  to  cite  those 
closest  to  the  effort  and  those  who  assisted  in  the  preparation  of  this  report.  The  original 
eight  PLATO  authors  were  MSgt  Carl  Dennis,  Sgts  Mel  Marcus,  Dennis  Mitts,  and  Roger 
Pelkey,  Ms.  Janice  Dallman,  Mr.  Bruce  lehl,  Mr.  William  Kennedy  and  Mr.  John 
Predmore.  Joining  the  project  later  were  Mr.  Ken  Burkhardt,  Mr.  Paul  Aschenbrenner,  Mr. 
Bob  Blocker,  Sgt  Richard  Bourne,  A1C  Jerry  Lynch,  Ms.  Georgia  Rodetis,  and  Ms.  Kathy 
Wigton.  Instructional  Systems  Development  personnel  who  designed  the  curriculum  and 
prepared  course  materials  other  than  PLATO  lessons  included  TSgt  Keith  Bloomer,  Mr. 
Don  Brown,  SSgt  James  Farren,  MSgt  Edward  Siedlemen,  MSgt  Owen  Smith  and  TSgt 
Thomas  Smith.  The  individual  who  integrated  the  efforts  of  the  ISD  and  PLATO 
personnel  into  a unified  project  was  Mr.  Robert  Gissing.  Various  individuals  assisted  in 
the  performance  of  the  evaluation:  TSgt  Ralph  Bush.  Sgt  Frank  Gaston.  Sgt  David  Gerts, 
SSgt  Ernie  Goenag,  Lt  Joseph  Green.  SSgt  Vance  Lennon.  Sgt  James  Mitchell,  TSgt 
Henry  O’Neil,  Sgt  Sherwood  Schoff,  SSgt  Michael  Stoll  and  Lt  Harold  Trask.  It  is 
expected  that  the  University  of  Illinois  Computer-Based  Education  Research  Laboratory 
personnel  will  be  acknowledged  in  their  reports,  but  a special  thanks  should  be  extended 
to  Mr.  Lynn  Misselt  and  Ms.  Kikumi  Tatsuoka  for  their  efforts  in  data  analysis  and  Mr. 
Joseph  Klecka  for  his  detailed  study  of  the  PLATO  lessons.  Likewise,  the  assistance  of 
Dr.  Jerry  Dcignan,  Amn  Eric  Duncan  and  Amn  Karen  Kozlowski’s  data  analysis  assistance 
at  the  Air  Force  Human  Resources  Laboratory  (AFHRL)  was  appreciated.  The  authors 
would  also  like  to  thank  Major  Brian  K.  Waters  (AFHRL)  for  his  extensive  technical 
review  and  substantive  comments,  and  Mrs.  Virginia  Bahnsen  for  the  numerous  hours 
expended  in  typing  and  proofreading  the  draft  report. 
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EVALUATION  OF  PLATO  IV  IN  VEHICLE  MAINTENANCE  TRAINING 


I.  INTRODUCTION 


Project  Genesis 

During  the  1960s,  technological  advances  made  at  the  University  of  Illinois’  Computer-based 
Education  Research  Laboratory  (CERL)  provided  substance  to  the  dream  of  computer-based  education 
which  someday  may  become  cost-effective.  This  new  technology  became  the  heart  of  a fourth  generation 
teaching  system  called  PLATO  IV  (Alpert  & Bitzer,  1970).  The  acronym  stands  for  Programmed  Logic  for 
/lutomated  Teaching  Operations.  Interest  in  the  computer-assisted  instruction  (CA1)  capabilities  of  PLATO 
grew  along  with  the  evolving  system.  Monitoring  this  evolution  for  potential  applications  within  military 
technical  training  environments  were  a number  of  government  agencies,  among  them  the  Defense  Advanced 
Research  Projects  Agency  (ARPA),  the  Technical  Training  Division  of  the  Air  Force  Human  Resources 
Laboratory  (AFHRL),  and  the  Air  Training  Command  (ATC).  In  1971  ARPA  agreed  to  fund  a tri-service 
test  of  the  PLATO  system.  The  Air  Force  test  commenced  in  1972  at  the  Chanute  Technical  Training 
Center,  Chanute  AFB,  Illinois.  Proximity  to  the  University  and  the  continuing  interest  expressed  by  the 
Training  Research  Applications  Branch  (TRAB)  at  Chanute  were  important  factors  in  the  decision  to  use 
Chanute  as  a field  test  site.  The  ensuing  ARPA/AFHRL/ATC  research  agreement  stated  that  the  project’s 
objective  was  “to  conduct  a small  scale  service  test  of  tire  PLATO  IV  system  in  a technical  training 
environment  to  detemiine  its  cost  effectiveness,  acceptance,  incorporability  (into  ongoing  technical 
training),  and  reliability.” 

Responsibilities 

ARPA  furnished  the  necessary  hardware,  and  by  means  of  a contract  with  CERL,  provided  training 
and  assistance  to  the  Chanute  test  site.  AFHRL’s  primary  obligations  were  to  provide  direction, 
consultation,  perform  monitorship  and  liaison  roles,  and,  in  conjunction  with  the  TRAB,  to  evaluate  and 
document  the  project  outcomes.  ATC  provided  manpower  and  facilities. 

Project  Personnel 

Project  manning  varied  considerably  over  the  service  test’s  run.  Since  changes  in  manpower  were 
extremely  important  to  the  outcome  of  the  service  test,  a detailed  discussion  is  included  in  the  section 
covering  site  history.  The  primary  project  personnel  consisted  of  an  on-site  AFHRL  project  monitor,  two 
TRAB  members,  eight  authors  from  the  Vehicle  Maintenance  Branch,  and  a computer  programmer. 

Scope  of  Development 

The  project  involved  the  four  Special  Purpose  Vehicle  Repairman  courses.  PLATO  was  used  in 
connection  with  a major  redesign  of  a portion  of  these  courses  shared  in  common.  This  effort  was 
performed  by  the  3340th  Technical  Training  Group’s  Instructional  Systems  Development  (ISD)  team  with 
PLATO  designated  the  delivery  system  for  35  lessons  in  29  subject  matter  areas. 


II.  BACKGROUND 


PLATO  System  Description 

During  the  service  test’s  operational  phase,  the  PLATO  system  (Alpert  & Bitzer,  1970)  consisted  of 
approximately  1,000  time-shared  terminals  connected  to  a Control  Data  Corporation  Cyber  73-24 
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dual-processing  computer  via  microwave  and  telephone  lines.  Most  were  centered  at  various  educational 
facilities  within  or  near  the  university  campus;  however,  terminals  were  distributed  throughout  the  United 
States  and  some  were  located  in  foreign  countries.  Though  many  hardware  features  were  important,  a 
significant  feature-one  that  allowed  hundreds  of  lessons  to  be  used  simultaneously  with  an  average 
response  latency  of  125  milliseconds-was  the  Extended  Core  Storage  (ECS)  unit.  This  electronic  memory 
permitted  the  two  processors  rapid  access  to  the  CA1  lesson  coding,  keeping-  response  latency  low,  i.e., 
students  viewed  changes  to  the  lesson  display  as  virtually  simultaneous  with  their  actions. 

Another  innovative  feature  was  the  plasma  panel  terminal  (Figure  1)  which  should  be  distinguished 
from  the  cathode  ray  tube  (i.e.,  television)  variety.  The  panel  consisted  of  a grid  of  conductors,  sandwiched 
by  two  glass  sheets  (8.5"  by  8.5"  by  .5"),  with  a neon  gas  mixture  filling  the  space  between  grid  and  glass. 
When  voltage  was  applied  to  any  combination  of  the  more  than  250,000  conductor  intersections,  the  neon 
mixture  provided  an  orange  glow.  Thus,  the  display  of  alphanumeric  characters  and  graphics  was  possible 
by  illuminating  adjacent  intersections.  Projection  of  a microfiche  image  on  this  panel  was  possible  due  to 
the  transparent  nature  of  the  panel.  Students  interacted  with  the  terminal  via  a typewriter  keyboard  or  by 
physically  touching  the  dispaly  screen.  This  touch-panel  detected  the  screen  location  a student  pointed  to; 
for  instance,  when  he  was  instructed  to  identify^  onp  automotive  part  -onafl  image  depicting  several 
alternatives. 

PLATO  lesson  authors  prepared  material  through  the  use  of  a special  simplified  programming 
language  known  as  TUTOR.  For  simple  displays  and  instructional  programming  strategies,  TUTOR  was 
effectively  employed  by  site  authors.  Programming  lessons,  collecting  data,  and  managing  student 
instructional  sequences  were  facilitated  by  a variety  of  generalized  programming  routines  developed  by 
CERL  personnel  or  other  PLATO  system  users. 

At  Chanute,  twenty-nine  terminals  were  used  for  training  students  and  authoring  lessons,  while  one 
terminal  was  dedicated  to  special  functions  such  as  monitoring  a student’s  interaction  with  a lesson, 
diagnosis  and  remediation  of  spurious  coding  difficulties,  and  communication  with  CERL  personnel. 
Student  terminals  were  located  in  a 25-  by  50-foot  room-each  terminal  in  its  own  carrel.  A microware  link 
was  used  to  transmit  data  from  CERL  to  Chanute,  while  a telephone  line  returned  information  from  site 
terminals  to  the  computer  mainframe. 

Historical  Perspective 

The  PLATO  IV  service  test  was  characterized  by  three  distinct  phases.  Phase  1 was  a prolonged  period 
of  hardware  acquisition  and  planning  in  which  the  definition  of  project  goals  and  philosophy,  and 
development  of  an  effective  management  scheme  were  the  two  major  events.  During  this  phase,  personnel 
selection  and  training,  equipment  acquisition,  and  lesson  development  were  also  accomplished.  Phase  I 
commenced  in  July  1972  and  ended  in  June  1974. 

Phase  II  marked  the  transition  of  the  project  to  an  operational  mode  with  emphasis  on  the 
development  of  CAI  materials  in  association  with  the  redesign  of  the  Special  Purpose  Vehicle  Repairman 
courses.  In  seven  months,  the  courseware  was  ready  for  the  first  class  tryout  (14  January  1975).  It  took 
approximately  one  additional  year  to  validate  all  PLATO-niediated  materials  and  obtain  the  bulk  of  the 
evalution  data. 

Phase  III  began  in  October  1975  and  concluded  in  June  1976.  The  original  agreement  with  ARPA 
called  for  a 30  September  1975  termination  date; however,  additional  time  was  required  for  evaluation  data 
collection.  ARPA  granted  the  extension,  with  the  provision  that  additional  development  goals  be  specified 
and  these  new  efforts  be  evaluated.  These  consisted  of  the  development  of  seven  experimental  lessons 
investigating  several  different  research  areas,  selected  additional  CAI  applications,  and  an  expanded  testing 
program. 

Phase  I.  A memorandum  of  understanding  between  ARPA,  AFHRL,  ATC,  and  the  Chanute  TRAB, 
drawn  in  June  1972,  established  the  general  orientation  of  the  service  test.  As  a result,  it  became  the 
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TRAB’s  responsibility  to  develop  a research  agreement  between  these  agencies,  to  commence  detailed 
planning  activities  in  conjunction  with  AFHRL,  and  to  gather  the  necessary  resources. 

The  early  project  documentation  indicated  that  a variety  of  courses  were  considered  as  targets  for 
PLATO  implementation,  with  the  final  selection  being  the  General  Purpose  Vehicle  Repairman  course. 
Primary  decision  criteria  were:  (a)  a non-con troveisial  nature  (important  at  the  time  because  of  university 
student  attitudes),  (b)generalizability  of  results  to  other  training  environments  such  as  community  colleges, 
and  (c)  a mechanical  rather  than  electrical/electronics  orientation.  The  significance  of  the  third  criterion 
may  not  be  readily  apparent.  Much  of  the  previous  work  with  CAI  systems  had  been  in  the 
electrical/electronics  area  where  cognitive  skills  aie  emphasized,  and  where  the  caliber  of  students  was 
exceptional.  It  was  hoped  that  PLATO— owing  to  its  simplified  lesson  development  and  usage 
features-could  be  e ily  assimilated  within  the  mechanical  training  environment.  This  would  be  a highly 
significant  finding  for  the  Air  Force,  since  much  ATC  training  has  a mechanical  orientation. 

After  selection  of  a target  course  and  following  agreement  on  a general  approach,  PLATO  lesson 
authors  were  chosen  from  the  Vehicle  Maintenance  Branch  instructor  personnel.  Four  military  and  four 
civilian  authors  were  selected  on  the  basis  of  education  and  productivity,  as  no  criteria  of  a more  specific 
nature  were  known  to  correlate  with  performance  as  CAI  authors.  In  general,  the  author  group  was  young, 
motivated,  and  well  educated  compared  to  the  other  vehicle  maintenance  instructors.  However,  the  group 
had  little  instructional  design  experience  and  six  selectees  were  relatively  inexperienced  as  instructors.  A 
detailed  description  of  the  original  eight  authors  is  available  in  Green  (1973).  None  of  the  eight  had  any 
previous  association  with  CAI  or  computer  programming.  Only  two  had  some  experience  with  lesson 
material  development.  Also,  this  group’s  classroom  instruction  experience  and  subject  matter  familiarity 
were  not  extensive.  A substantial  amount  of  training  time  was  invested  in  this  group  before  they  were 
considered  qualified  authors. 

Initial  TUTOR  language  training  was  presented  by  CERL  representatives.  Though  some  CAI 
courseware  development  discussions  were  included  in  this  program,  CERL  had  anticipated  that  author 
selectees  would  be  knowledgeable  in  the  area  of  curriculum  development  and  would  merely  require 
specialized  training  on  CAI  techniques.  CERL  provided  supplementary  informal  lesson  development 
training  through  periodic  site  visits  from  representatives.  In  addition,  authors  were  required  to  read  two 
CAI-related  articles  per  week.  This  cultivation  of  the  author  group’s  skills  was  a long  and  tedious  process 
due  to  the  amount  to  be  learned,  the  lack  of  proven  guidelines,  the  trainer’s  and  trainees’  inexperience  with 
CAI,  and  the  changeability  of  the  PLATO  IV  system  during  its  early  development. 

Lesson  preparation  efforts  during  Phase  I were  directed  at  producing  innovative  materials,  taxing  the 
capabilities  of  system  and  authors  alike.  Each  author  became  an  investigator  in  his  own  right,  attempting  to 
duplicate  eye-catching  instructional  strategies  and  incorporate  his  own  ideas.  Yet,  individual  capabilities  and 
interests  soon  sorted  authors  into  specialized  roles.  Those  who  were  not  interested  or  talented  in  developing 
lesson  content  centered  their  efforts  on  computer  programming.  Those  who  didn’t  have  the  interest  or 
aptitude  to  struggle  with  the  complex  coding  (needed  for  the  more  innovative  lessons)  attended  to 
instructional  design  or  curriculum  development.  One  author  spent  most  of  his  time  on  hardware-related 
activities.  These  first  experiences  were  quite  valuable  in  determining  the  future  operational  structure,  when 
the  project’s  original  research  orientation  changed  to  one  of  operational  efficiency. 

System  hardware  and  software  modifications  were  frequent  during  the  first  years.  Along  with  several 
other  recurring  problems,  these  changes  tended  to  inhibit  author  training  and  lesson  preparation  activities. 
Prolonged  equipment  delivery  slippages,  ECS  memory  shortages,  and  system  unreliability,  reduced  terminal 
availability  for  authoring  and  precluded  student  tryout  of  the  new  courseware.  Such  circumstances  were 
extremely  frustrating  to  the  Chanute  authors. 

Attrition  of  personnel  greatiy  'limited  productivity  during  the  first  stage  of  the  project.  Despite  verbal 
agreements,  two  military  authors  left  the  service  before  the  end  of  their  commitment.  In  their  year  with  the 
project,  each  had  demonstrated  considerable  talent  in  the  development  of  lessons,  and  their  loss  was 
unfortunate,  especially  since  the  acquisition  and  training  of  a replacement  often  took  from  six  months  to  a 
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year.  To  increase  work  force  stability,  and  to  provide  computer  programming  assistance  to  remaining 
authors,  a civilian  computer  programmer  was  hired  in  February  1974.  This  acquisition  facilitated 
courseware  production,  since  the  tedious  and  complex  coding  was  done  more  efficiently. 

By  late  1973,  an  adequate  service  test  plan  was  yet  to  be  prepared.  Service  test  conditions  had 
stabilized;  consequently,  the  potential  for  successful  execution  was  great,  yet  some  major  deficiencies  were 
apparent,  and  ARPA,  AFHRL,  and  ATC  representatives  convened  to  address  the  problems.  It  was  decided 
that  project  management  required  strength,  experience,  and  above  all,  adequate  support.  The 
representatives  recommended  that  the  project  become  the  responsibility  of  the  3340th  Technical  Training 
Group’s  ISD  team. 

The  important  outcomes  of  a May  1974  project  review  were  a clear,  new  definition  of  project 
orientation,  and  an  organizational  structure  with  sufficient  support  to  achieve  the  new  goals.  The  earlier 
research  orientation  was  gone;  PLATO  was  considered  to  be  an  operational  training  tool,  part  of  an  arsenal 
of  devices  employed  by  the  ISD  team.  Management  was  centralized  and  the  ISD  team  was  made  responsible 
for  all  development  functions.  A manifold  increase  in  project  interest  was  evident  at  all  levels. 

Phase  //.  Changing  the  project’s  orientation  to  an  operational  setting  was  consistent  with  the  service 
test’s  primary  goals.  Yet,  by  the  project’s  end,  it  was  recognized  that  research  to  determine  CA1  application 
had  been  sacrificed.  Detailed  planning  prepared  by  the  ISD  team,  during  June  1974,  called  for  the  new 
course  tryout  to  commence  in  January  1975.  PLATO  would  serve  as  the  principal  medium  for  33 
knowledge-oriented  objectives.  Hence,  in  six  months,  16  new  lessons  were  prepared,  and  17  existing  lessons 
were  transformed  to  coincide  with  the  objectives  of  the  new  system. 

In  the  ISD  team’s  management  plan,  a principal  objective  was  to  “Make  maximum  use  of  PLATO’s 
CAI  and  CMI  capabilities.”  As  used  at  Chanute,  the  term  CAI  (Computer-zlssistcd /nstruction)  referred  to 
materials  which  presented  information  to  the  student  for  learning  purposes;  i.c.,  student  lesson  materials. 
This  should  be  distinguished  from  CMI  (Computer-A/anaged  /rrstruction)  which  can  include  student 
evaluation  administrative  functions,  adaptive  lesson  sequencing  for  individual  or  group  needs,  and  extensive 
data  collection.  CAI  efforts  were  directed  at  the  33  lessons,  subject  matter  being  the  underlying  facts  and 
principles  necessary  to  perform  special  vehicle  maintenance  tasks.  CMI  uses  included  test  administration, 
student  curriculum  routing,  automated  data  collection,  and  clerical  functions. 

To  prepare  CAI  courseware,  the  ISD  team  provided  the  PLATO  authors  with  lesson  objectives, 
objective  tests,  and  teaching  points.  ISD  personnel  closely  monitored  author  activities,  since  none  of  them 
were  fully  qualified  in  all  three  facets  of  a CAI  author’s  role  (i.e.,  computer  coder,  subject  matter  expert, 
and  instructional  programmer).  Thus,  the  lessons  were  developed  using  a quasi-team  approach.  After  an 
adjustment  period,  this  procedure  worked  reasonably  well,  and  on  15  January  1975  the  first  class  tryout  of 
all  new  courseware  commenced. 

A variety  of  PLATO  system  characteristics  retarded  lesson  development  and  validation  progress. 
Preparing  microfiche  for  PLATO’s  slide  projection  capability  proved  to  be  a complex,  time-consuming 
process.  ECS  limitations-imposed  on  each  PLATO  site  when  memory  approached  capacity— limited  the 
number  of  authors  that  could  simultaneously  program  materials.  The  instructor/author  found  that  the  more 
intricate  CAI  lesson  strategies  required  sophisticated  programming  skills.  Many  of  the  authors  never  did 
attain  the  required  proficiency  level. 

Overall,  Phase  II  was  regarded  as  the  most  productive  and  valuable  period,  and  much  of  the  acquired 
experience  should  be  applicable  to  future  CAI  endeavors  in  a military  technical  training  context. 

Phase  HI.  Phase  III  started  on  1 October  1975  and  concluded  30  June  1976,  although  some  Phase  II 
activities  (lesson  validation)  were  ongoing  until  January  1976.  In  addition,  some  Phase  III  activities 
continued  into  the  ATC  funded  extension  of  the  service  test.  During  Phase  III,  the  three  basic  objectives  of 
a revised  management  plan  were  to  be  accomplished,  along  with  continuing  evaluation  of  the  PLATO-bascd 
vehicle  maintenance  courses.  The  first  objective  was  to  investigate  CAI  applications  through  three  successive 
levels  of  studies.  Seven  priority  A studies  described  in  the  plan,  sought  to  derive  basic  techniques  which 
would  have  applications  to  technical  training.  In  general,  B-  and  C-lcvel  studies  were  for  the  purpose  of 
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refining  the  more  promising  techniques  noted  in  the  A studies.  The  second  objective  was  an  expanded  use 
of  CMI,  predominantly  in  the  testing  area.  A third  objective  sought  to  identify  and  implement  CA1 
applications  for  the  latter  portion  of  each  Special  Purpose  Vehicle  Repairman  course. 

During  the  planning  stage,  these  objectives  did  not  seem  unrealistic.  By  the  end  of  the  service  test, 
however,  only  one  of  the  seven  studies  was  completed,  CMI  applications  were  partially  accomplished,  and 
further  application  of  CA1  was  not  attempted.  1SD  and  PLATO  personnel  losses  devastated  this  part  of  the 
project,  since  only  one  of  the  original  eight  authors  remained  by  March  1976.  Until  their  departure, 
experienced  authors  were  used  primarily  to  finish  Phase  II  activities  such  as  lesson  validation  and  microfiche 
preparation.  The  operationally  oriented  ISD  team  felt  comfortable  with  plans  of  their  own  design;  they  did 
not  approve  of  exploratory  research  planned  by  others.  As  a result,  departures  from  the  original  plans  were 
numerous  and  the  priority  for  completion  became  secondary.  As  in  Phase  I objectives,  planning, 
management,  authority,  and  responsbility  were  not  clearly  defined.  In  the  face  of  a heavy  workload  and 
uncertainty  over  the  project’s  future,  achieving  the  planned  objectives  became  impossible. 

What  was  completed  during  Phase  III  owed  its  existence  to  a 3 March  1976  program  review.  The 
purpose  of  the  review  was  to  consider  preliminary  evaluation  data  and  to  formulate  future  directions  for 
the  project.  It  was  apparent  to  the  technical  school  and  ATC  managers  that  two  additional  cost-related 
studies  were  warranted.  As  a result,  ATC  funded  a short  extension.  The  high  level  interest  in  the  cost 
studies  carried  over  to  the  ARPA  goals,  and  was  the  principal  factor  in  their  partial  completion. 

The  elementary  consideration  in  the  service  test  was  to  describe,  document,  and  evaluate  PLATO's 
incorporability  into  Air  Force  technical  training.  In  spite  of  numerous  difficulties,  the  test  site  did 
accomplish  this  basic  mission. 

Instructional  Setting 

Describing  the  features  of  the  Special  Purpose  Vehicle  Repairman  courses  (before  and  after  the  ISD 
curriculum  modifications)  will  assist  the  reader  in  understanding  the  evaluation  design.  The  four  courses 
and  their  respective  content  areas  are  depicted  in  Figure  2.  Each  of  the  courses  was  termed  a “shred-out” 
and  varied  in  length  according  to  the  complexity,  or  number  of  vehicles,  within  a shred  classification.  For 
example,  the  “A  Shred”  or  Crash/Fire  Vehicles,  had  a 16-week  duration  in  its  “conventional”  form.  The 
term  conventional  will  be  used  in  reference  to  the  pre-lSD  course  version. 

All  conventional  courses  were  divided  into  instructional  units  called  blocks.  The  first  four  blocks  of 
each  course  were  identical;  that  is,  the  information  and  laboratory  skills  were  common  to  the  four  shreds. 
Blocks,  which  followed  the  common  core  segment,  presented  information  unique  to  a specific  class  of 
vehicles.  Since  all  students  received  instruction  in  the  common  segment,  student  flow  was  highest  here, 
about  ten  per  week. 

Students  received  instruction  6 hours  a day,  5 days  a week,  during  the  6 a.ni.  to  noon  time  period.  On 
a typical  day,  the  instructor  might  lecture  for  4 hours  covering;  for  example,  principles  of  carburetor 
operation;  then  students  would  spend  the  remaining  2 hours  performing  carburetor  maintenance  tasks  in 
the  laboratory.  This  knowledge  to  performance  ratio  was  60  percent  to  40  percent  in  the  conventional 
courses.  The  courses  were  lock-step,  meaning  that  the  instructor  adhered  to  a time  schedule  specified  in  the 
course  control  documents. 

Student  evaluation  was  derived  from  instructor  observation  and  block  tests.  Every  day,  the  instructor 
made  a qualitative  decision  concerning  an  individual’s  composite  performance.  A student  could  fail  a lesson 
test,  but  do  well  in  class  and  laboratory  activities,  and  receive  a satisfactory  performance  rating.  This 
procedure  was  one  of  the  determinants  for  assigning  specialized  individual  assistance  (SIA)  to  the  students, 
as  were  absenteeism  and  performance  on  comprehensive  block  tests.  End-of-block  tests  generally  consisted 
of  approximately  30  to  40  multiple-choice  questions.  Students  who  failed  were  either  “washed-back”  to 
the  following  class,  given  a probationary  continuation  (allowed  to  continue  with  their  class  pending  the 
results  of  a retest)  or  eliminated  from  the  course  if  several  successive  failures  had  occurred. 
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The  ISD  team  substantially  changed  the  Special  Purpose  Vehicle  Repairman  courses.  In  refining  an 
existing  course,  an  ISD  team  applied  a specific  procedural  model  (AFP  50-58,  1974).  Scope  and  depth  of 
training  received  emphasis  to  insure  that  the  end  product  was  relevant  to  field  needs  and  efficient.  In  each 
of  the  Special  Purpose  Vehicle  Repairman  shred  areas,  subjects  were  added  or  deleted,  and  the  extent  of 
task  training  increased  or  decreased  to  reflect  the  newly  determined  course  requirements. 

Numerous  other  changes  occurred.  The  basic  design  became  group-paced,  wherein  a class  of  students 
progressed  through  their  course  (common  course  segment  and  the  particular  shred  area  assigned  to  that 
class)  at  the  rate  of  the  slower  students.  Individualized  media;  such  as  PLATO,  programmed  texts,  and 
single  concept  films,  were  created  for  the  common  course  segment,  with  group  lectures  the  primary  delivery 
mode  in  the  shred  areas.  The  self-paced  nature  of  individualized  media  led  to  variable  student  completion 
times.  While  in  the  PLATO  terminal  room,  faster  students  interacted  with  enrichment  materials  until  the 
slower  students  caught  up.  These  enrichment  lessons  consisted  of  material  which  had  been  developed  for 
the  General  Purpose  Vehicle  Repairman  course,  as  well  as  related  vehicle  maintenance  lessons  selected  from 
the  curricula  of  other  PLATO  users;  they  were  not  however,  a part  of  the  mainstream  special  purpose 
materials. 

As  another  substantive  change,  the  “Block  X”  concept  was  employed  for  autonomous  subject  matter. 
Lessons  which  bore  little  positional  importance  in  the  hierarchy  of  vehicle  maintenance  subject  matter  were 
given  as  work  to  be  done  during  the  complementary  technical  training  period,  or  in  other  words, 
“home-study.”  Students  were  given  a Block  X examination  after  completing  Block  IV. 

The  new  courseware  had  an  impact  on  the  role  of  the  instructor  in  the  common  course  segment  and 
the  knowledge/performance  ratio.  The  instructor  became  more  of  a manager  of  instruction,  observing  and 
assisting  students  on  an  individual  basis  while  they  interacted  with  PLATO,  or  performed  laboratory  tasks. 
Another  important  change  was  the  reversal  of  the  knowledge/performance  ratio;  that  is.  40  percent  of  the 
students’  time  was  devoted  to  background  knowledge  and  60  percent  to  laboratory  task  skills.  Of  this 
background  knowledge,  virtually  all  was  presented  via  PLATO;  handtools  and  technical  orders  being 
exceptions. 

Some  differences  in  student  evaluation  between  the  conventional  and  ISD  course  versions  were 
noteworthy.  Each  PLATO  lesson  had  an  end-of-lesson  test.  Upon  failure,  students  were  automatically 
routed  to  the  beginning  of  a lesson,  or  in  some  cases,  just  to  the  weak  areas  of  the  lesson.  Block  tests  were 
rewritten  to  account  for  the  instructional  modifications  and  were  administered  via  PLATO.  Students 
received  the  test  critique  via  PLATO  instead  of  in  a group  session  overseen  by  the  instructor.  PLATO 
student  shift  assignments  were  influenced  by  system  maintenance  hours.  Students  received  instruction  from 
noon  until  6 p.ni.  as  the  PLATO  system  was  not  available  until  after  7:40  a.m. 

A brief  description  of  the  PLATO  lessons  will  be  provided;  the  reader  who  desires  a more  substantive 
examination  is  referred  to  the  Instructional  Materials  Development  portion  of  this  report  (Section  VI).  The 
majority  of  PLATO  courseware  resembled  the  tutorial  lesson  strategy  discussed  by  Levien  in  1972. 
Information  was  presented  in  one  screen  display  or  in  a sequence  of  displays,  and  then  student  learning  was 
assessed.  Typical  test  item  formats  were  multiple-choice,  matching,  or  true/false.  Contingency  branching 
was  at  times  employed;  however,  most  lessons  merely  required  the  student  to  get  the  right  answer  before 
progressing. 

Selecting  an  Evaluation  Paradigm 

When  planning  for  an  evaluation,  it  is  essential  that  the  evaluators  consider  the  needs  of  the  audience 
and  the  purpose  of  the  evaluation.  An  appropriate  perspective  and  evaluation  design  may  then  be  selected. 
For  the  present  study,  a survey  was  sent  to  ATC,  ARPA,  AFIIRL,  CERL  and  other  military  PLATO  sites. 
The  survey  requested  these  agencies  to  rank -order  six  basic  evaluation  areas  and  likewise  order  questions 
within  each  area.  The  criteria  for  ranking  were  the  importance  of  the  area  or  question  to  that  agency’s 
needs.  This  exercise  confirmed  the  notion  that  different  needs  and  project  expectations  would  be  found. 
For  example,  the  project  evaluation  might  provide  decision  makers  with  information,  or  establish  a 
repository  for  CAI  research  and  experience.  However,  a decision  to  continue  or  not  continue  involvement 


with  the  PLATO  system  was  ATC’s  paramount  consideration  and  responsibility.  The  final  evaluation 
planning  mirrored  the  ATC  priorities,  stressing  the  accumulation  of  information  on  which  to  base  future 
PLATO-related  decisions  at  Chanute.  Systematic  investigation  of  various  CAI  realms  was  not  a major  goal 
of  this  evaluation. 

Evalution  activities  concentrated  on  cost,  instructional  effectiveness,  and  instructional  impact 
(attitudes)  because  of  their  importance  as  indicated  by  the  survey.  Instructional  materials  development, 
management,  and  human  factors  considerations  were  important,  but  had  a lower  priority.  With  regard  to 
cost,  expenses  dealing  with  equipment,  personnel,  operational  and  communication  were  examined  in 
relation  to  savings.  Instructional  effectiveness  included  the  study  of  key  course  and  PLATO 
medium-associated  variables  that  provided  a picture  of  student  performance,  course  efficiency,  PLATO’s 
performance  contribution,  et  cetera.  Instructional  impact  looked  at  student  and  instructor  attitudes, 
training  capabilities,  and  organizational  features  which  may  have  been  affected  by  PLATO’s  incorporation 
into  technical  training. 

Two  program  evaluation  paradigms  were  considered  applicable  to  the  PLATO  service  test.  The 
absolute  suggested  by  Cronbach  (1963)  involves  the  fonnal  study  of  a well-defined  student  sample  with 
respect  to  goal  attainment  and  side  effects.  This  model  does  resolve  several  of  the  internal  validity  issues 
present  when  a comparative  field  design  is  employed;  i.e.,  the  problem  of  equating  control  and  treatment 
groups.  It  also  avoids  the  curriculum  evaluation  enigma  Cronbach  refers  to  as  equivocal  results.  When 
comparing  separate  curricula,  it  frequently  happens  that  the  effectiveness  of  about  half  the  modules  in  each 
curriculum  will  be  greater  than  their  counterparts.  When  this  occurs,  no  superiority  for  either  curriculum 
can  be  demonstrated,  and  the  results  of  the  evaluation  are  equivocal. 

Comparative  design  limitations  have  been  noted  by  many  evaluation  theorists  (Glass,  1972,  p.  105), 
but  as  Scriven  ( 1 967)  pointed  out,  in  terms  of  the  decision-making  purpose  of  most  evaluations, 
comparisons  are  often  unavoidable.  When  an  evaluation  serves  as  the  basis  for  selecting  one  program  over  a 
competing  one,  comparing  the  attributes  of  each  is  a necessity.  From  ATC’s  perspective,  many  instructional 
system  versions  would  allow  students  to  attain  training  goals;  however,  only  the  most  efficient,  economical 
version  was  desirable.  Efficiency  had  to  be  underscored  because  of  its  direct  relationship  to  training  costs. 
Determining  the  relative  efficiency  of  conventional  and  PLATO-based  course  versions  was  a high  priority 
ATC  need.  Hence,  the  comparative  paradigm  was  chosen  to  examine  most  of  the  evaluation  questions. 

The  comparative  design  had  to  maximize  the  interpretability  of  the  findings.  If  the  primary 
performance  outcomes  of  the  before-and  after-course  versions  were  compared,  the  conclusions  reached 
would  pertain  to  the  course  as  a whole,  not  to  a particular  medium.  Statements  concerning  the  specific 
contribution  of  a medium,  design  technique,  or  any  particular  curriculum  change  would  not  be  justified  by 
comparing  block  examination  scores,  completion  times,  and  failure  rates  because  many  features  of  the 
courses  had  changed.  Since  the  present  evaluation  was  chiefly  interested  in  PLATO’s  contribution  to  the 
instructional  environment,  the  evaluation  method  had  to  attend  to  the  problem  of  multiple  changes. 
Furthermore,  it  was  important  that  media  comparisons  involve  similar  classes  of  media  under  varying  types 
of  instructional  circumstances. 

A number  of  investigators  have  attempted  to  ascertain  CAl’s  contribution  compared  to  other  media. 
In  separate  reviews  of  CAI  projects  and  studies,  Taylor  (1974)  and  Jamison,  Suppes,  and  Welles,  (1974) 
cited  results  indicating  that  CAI-based  curriculum  students  performed  as  well  as  or  better  than  students  in 
the  non-CAl  (control)  curriculum.  Time  savings  were  typically  present  in  favor  of  CAI,  but  several 
questions  remained  unanswered.  What  was  CAI’s  direct  contribution?  Were  other  changes  present  that 
could  account  for  the  greater  efficiency?  The  comparison  groups  in  almost  every  study  were  conventional 
classroom -lecture  settings.  Comparing  almost  any  self-paced  or  group-paced  setting  with  a group  lock-step 
situation  would  yield  these  results,  since  the  first  two  make  use  of  individualized  media  and  the  latter, 
group  instruction.  Therefore,  interpretation  of  relative  instructional  efficiency  findings  was  limited  in  those 
projects  and  studies. 
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Other  studies  have  attempted  to  control  for  contaminating  influences,  thus  increasing  the  precision  of 
the  results.  In  one  study  (KPR  73-1  18,  1974)  at  the  USAF  School  of  Applied  Aerospace  Sciences,  Kcesler 
Air  Force  Base,  Mississippi,  programmed  test  (PT)  versions  of  lessons  developed  for  the  Lincoln  Terminal 
System  (LTS)  were  prepared  by  the  authors  of  the  original  lessons.  Care  was  taken  to  change  only  those 
aspects  of  the  lessons  necessary  to  prepare  an  off-line  version.  Tire  PT  lessons  were  then  administered  to  a 
sample  of  student  front  the  same  electronics  course  as  the  LTS  project.  Result  indicated  that  achievement 
scores  were  significantly  greater,  but  completion  times  of  the  LTS  students  were  significantly  longer  than 
those  of  the  PT  students.  The  investigators  concluded  that  though  achievement  scores  were  better,  the  LTS 
instruction  was  less  efficient.  In  another  study  a comparison  was  made  at  the  lesson  level  between  student 
groups  in  an  Army  machinist  training  course  (Dept  of  the  Army,  1975).  The  course  was  self-paced  using 
individualized  media  such  as  single-concept  films,  programmed  texts,  and  sound-on-slidc  presentations. 
PLATO  lessons  were  prepared  concurrently  for  some  of  the  instructional  objectives.  Students  were 
randomly  assigned  to  a PLATO  or  an  alternative  medium  lesson  condition,  so  that  most  students  received 
several  PLATO  lessons.  In  general,  the  PLATO  versions  were  as  effective  as  the  alternative  versions,  and 
produced  time  savings. 

It  can  be  seen  that  many  factors  influence  the  clarity  of  evaluation  findings.  The  present  evaluation 
was  structured  to  be  responsive  to  these  factors  by  being  flexible  in  its  choice  of  methodologies. 

Overall  Methodology 

The  current  evaluation  concentrated  on  three  arcas-instructional  effectiveness,  instructional  impact, 
and  cost.  The  method  was  comparative  and  primarily  directed  at  variables  which  would  reveal  course  and 
medium  differences.  Some  methodology  was  common  to  the  three  areas,  but  the  design  used  in  each  area 
was  dependent  upon  the  nature  of  the  area,  available  resources,  and  the  limitations  placed  on  the  evaluation 
team.  This  section  presents  those  evaluation  methods  common  to  the  examination  of  the  three  areas. 

Conditions.  The  first  of  the  four  conditions  was  termed  baseline  (BL).  The  purpose  of  the  BL 
condition  was  to  determine  major  course  parameters  before  ISD  interaction  changed  the  course. 
Administrative  records  for  a 9-month  period  prior  to  14  January  1975  were  reviewed  and  relevant  course 
data  extracted. 

Similar  to  the  BL  condition,  the  non-PLATO  (NP)  condition  served  as  a control  group.  It  existed 
concurrently  with  the  remaining  PLATO  conditions,  but  consisted  only  of  A-shred  students.  Students 
within  this  conventional  setting  knew  of  PLATO’s  existence  and  that  evaluation  data  were  being  collected. 
Evaluation  instruments  prepared  during  the  BL  condition  data  collection  period  were  administered  to  NP 
students. 

The  conventional  PLATO  (CP)  condition  was  composed  of  C-  and  D-shred  students  who  used  PLATO 
lessons  in  lieu  of  instructor  presentations.  Since  these  lessons  reflected  the  modifications  imposed  by  the 
ISD  training  analysis,  and  most  remained  unvalidated  throughout  the  duration  of  the  CP  condition, 
instructors  filled  in  any  gaps  they  perceived  with  supplemental  or  reinforcing  instruction.  The  amount  of 
PLATO  material  used  in  this  condition  was  variable,  ranging  from  about  70-  to  90-pcrcent  of  the  new  CAI 
lessons. 

The  remaining  condition  consisted  of  the  four  ISD  course  versions  and  was  termed  PLATO-based 
(PB).  In  this  condition,  students  interacted  with  all  new  courseware  in  the  context  of  an  updated  Special 
Purpose  Vehicle  Repairman  course  curriculum.  The  individual  shred  tryouts  were  staggered  because  of  the 
necessity  to  match  the  shred  area  instruction  to  the  common  course  segment.  As  soon  as  a shred  area 
revision  was  complete,  the  students  in  that  course  would  receive  the  new  materials.  Thus,  the  tryout 
commencement  dates  were:  15  January  1975  (B),  13  March  1975  (D),  22  May  1975  (C),  and  13  June 
1975(A). 

Student  Population.  Included  in  this  study  were  426  students  assigned  to  the  four  Special  Purpose 
Vehicle  Repairman  courses  for  training.  The  number  of  students  in  each  condition  was:  200  for  BL,  23  for 


16 


NP.  46  lor  C'P,  ami  1 57  for  PB.  Condition  sample  size  was  a function  of  the  staggered  tryout  schedule  for 
the  NP  and  CP  conditions.  In  most  cases,  deviations  in  sample  sizes  found  in  the  following  tables  were  the 
result  of  missing  observations. 


III.  INSTRUCTIONAL  I 1 1 K TIVLNKSS 

Tire  evaluation  of  instructional  effectiveness  focused  upon  three  areas:  the  course,  PLATO  as  a 
medium,  and  PLATO  courseware.  Procedures,  results,  and  discussion  will  be  discussed  for  each  area.  Overall 
conclusions  are  presented  later. 

Special  Purpose  Vehicle  Repairman  Course 

S/Hcial  I’’ trposc  Vehicle  Re /urn  nan  Course  I Voted  tires.  Primary  indices  of  instructional  effectiveness 
at  the  course  icvel  were  student  block  examination  grades,  course  completion  times,  eliminations, 
washbacks.  probationary  continuations,  and  special  individualized  assistance  time.  Tire  evaluation  team 
members  extracted  these  data  from  course  records  of  students  who  received  instruction  during  the  9- 
month  period  prior  to  1 5 January  I ‘>75  ( BL  condition)  and  from  I 5 January  1 ‘>75  to  30  September  1975 
(NP.  CP.  and  PB  conditions). 

Additional  data  were  provided  through  a field  follow-up  survey.  Supervisors  of  technical  school 
graduates  were  periodically  surveyed  by  the  Technical  Training  evaluation  (TTL)  Division,  Chanutc  APB, 
Illinois  to  determine  it  the  quality  of  those  graduates  was  meeting  An  f orce  needs.  Special  Purpose  Vehicle 
Repairman  (SPVR)  graduates,  from  classes  participating  in  the  PLATO  service  test,  were  surveyed  in  March 
1976.  These  individuals  were  from  classes  graduating  between  March  1975  and  October  1975.  Types  of 
data  collected  from  this  field  evaluation  arc  presented  in  Table  I.  Thirty-three  tasks  (in  this  particular 
functional  area)  were  common  to  all  four  shreds.  Tuese  tasks,  listed  in  Appendix  A.  were  tire  only  ones 

Table  I.  Sample  of  Field  Evaluation  Data 


Frequency  of  Graduates 
Performing  at  Level 


Level 

STS* 

Code 

Level 

Percent 

Qualified 

Percent 

Performing 

STS  Survey 
Qualified 
Average 

Task 

NP 

0 

1 

2 

3 

4 

Engines 

Determine  proper  operation  of  engines 

l 

0 

10 

51 

41 

13 

2 

91 

99 

2.5 

Engine  systems  and  components: 

Repair 

2 

0 

10 

52 

41 

11 

2 

91 

98 

2.4 

Service 

2 

0 

3 

37 

58 

14 

2 

97 

98 

2.7 

Test  and  adjust  engine  system  components 

0 

1 

12 

37 

43 

8 

2 

87 

100 

2.4 

Level  Interpretation 

4 Highly  Proficient.  Can  do  the  complete  task  quickly  and  adequately.  Can  tell  others  how  to  do  the 
task. 

3 Competent.  Can  do  all  parts  of  the  task.  Needs  only  a spot  check  of  completed  work.  Meets  minimum 
local  demands  for  speed  and  accuracy. 

2 Partially  Proficient.  Can  do  most  parts  of  the  task.  Needs  help  only  on  hardest  parts.  May  not  meet 
local  demands  for  speed  or  accuracy. 

I Extremely  Limited.  Can  do  simple  parts  of  the  task.  Needs  to  be  told  or  shown  how  to  do  most  of 
the  task. 

0 Incapable.  Can  do  no  part  of  the  task  without  being  told  or  shown  how  to  do  it. 

NP  Not  Performed.  Has  not  performed  task  in  current  assignment. 


Specially  training  standard. 
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analyzed.  In  addition  to  data  from  the  graduates  of  1975,  data  from  an  earlier  evaluation  of  course 
graduates  conducted  in  1973  were  available  for  use  as  a baseline.  These  data  were  examined  to  determine  if 
changes  had  occurred  in  the  student  population. 

Comparison  of  overall  performance  ratings  was  made  for  five  groups  of  graduates.  Those  from  CP, 
NP,  and  PB  conditions  constituted  three  groups.  Graduates  who  had  participated  in  PLATO  training  after 
all  classes  had  transitioned  to  the  experimental  system  (designated  Late  PLATO  or  LP)  made  up  the  fourth 
group,  and  the  1973  graduates  constituted  the  fifth,  baseline  (BL  73)  group.  (Note:  These  baseline  students 
were  not  the  BL  Group  of  earlier  analyses.) 

Results 

Block  Examination  Scores.  The  mean  common  course  segment  block  examination  scores  and 
standard  deviations  for  each  condition  are  presented  in  Table  2.  BL,  NP,  and  CP  scores  were  analyzed  by  a 
one-way  analysis  of  variance  (ANOVA)  design  for  each  of  the  common  course  segment  blocks  (Table  3).  No 
significant  differences  between  group  performance  were  found.  PB  scores  were  not  included  in  the  ANOVA 
design-even  though  they  were  apparently  greater  than  those  within  other  conditions-because  of  the 
aforementioned  block  examination  differences. 


Table  2.  Common  Course  Segment  Block  Examination  Data 


Block 

1 

2 

3 

4 

Conditon 

M SD 

M 

SD 

M 

SD 

M 

SD 

BL 

(N=190) 

79.0  10.9 

76.2 

10.3 

75.2 

12.4 

74.0 

9.8 

NP 

(N=23) 

76.5  10.3 

77.1 

9.6 

78.7 

10.7 

76.7 

9.2 

CP 

(N=46) 

76.6  11.2 

78.1 

9.5 

78.4 

12.3 

76.6 

10.5 

PBa 

(N=l  56) 

91.0  7.8 

80.9 

11.2 

83.3 

11.3 

86.3 

9.7 

“The 

block  examinations  for  this 

condition  are  different  than  the  other  conditions. 

Table  3.  ANOVA  Summary  Tables  for  Common  Course  Segment 

Block  Examination  Scores  (BL,  NP,  and  CP  Conditions) 

Block 

Source 

df 

MS 

F 

P 

l 

Condition 

2 

84.62 

.71 

.49 

Within  Cells 

253 

118.86 

2 

Condition 

2 

38.16 

.37 

.69 

Within  Cells 

253 

102.12 

3 

Condition 

2 

157.78 

1.05 

.35 

Within  Cells 

251 

149.89 

4 

Condition 

2 

98.05 

1.01 

.37 

Within  Cells 

246 

97.50 
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Course  Compeletion  Time.  Mean  course  completion  time  data  are  presented  in  Table  4.  Time  savings, 
resulting  from  the  group-paced  PB  condition,  were  computed  by  determining  the  difference  between  PB 
and  BL  conditions.  Table  5 shows  that  the  time  savings  for  the  common  course  segment  was  about  28.5 
percent.  An  estimate  of  additional  time  savings  attributable  to  PLATO  as  a self-paced  medium  will  be 
provided  later. 

Table  4.  Mean  Course  Completion  Times  (Hours) 


Shnd/Conditlons 


Block 

A 

B 

c 

o 

BL 

N = 3* 

NP 

N = 23 

P8 

N = 13 

BL 

N = 35 

PB 

N * 37 

BL 

N = 57 

CP 

N = 30 

PB 

N = 21 

BL 

N * 50 

CP 

N = 1* 

PB 

N s 41 

i 

57.6 

59.0 

42.6 

57.3 

51.1 

58.8 

57.7 

53.3 

56.8 

57.1 

54.7 

2 

58.2 

59.7 

43.7 

56.8 

45.7 

59.3 

56.3 

49.1 

51.5 

58.4 

52.5 

3 

60.6 

60.0 

36.8 

60.9 

43.9 

58.7 

61.9 

46.1 

59.7 

60.5 

42.8 

4 

59.8 

60.7 

21.4 

57.9 

25.3 

59.5 

56.8 

24.4 

58.6 

58.0 

24.0 

5 

59.4 

64.5 

68.7 

59.6 

45.4 

57.1 

59.4 

65.5 

59.1 

60.0 

61.4 

6 

59.0 

62.7 

57.7 

60.4 

67.7 

57.4 

59.6 

59.9 

60.1 

60.6 

31.8 

7 

58.3 

61.4 

60.0 

— 

— 

57.7 

59.9 

a— 

59.6 

61.2 

61.6 

8 

58.2 

56.7 

24.0 

- 

- 

- 

- 

- 

- 

- 

- 

Total 

471.1 

484.7 

354.9 

352.9 

279.1 

408.5 

411.6 

298.3 

405.4 

415.8 

328.8 

aC -shred  course  was  shortened  one  block. 


Table  5.  Mean  PB  Time  Saving  Relative  to  the  BL  Condition 

(llou rs  and  Percent  Reduced) 


Shred  Course 


Instruction 

A 

(N  = 13)* 

B 

(N  = 37) 

c 

(N  = 21) 

D 

(N  = 41) 

Common  Course  Segment 

22.9 

16.8 

15.8 

14.7 

38.8% 

28.8% 

26.8% 

23.2% 

Overall 

126.2 

68.5 

104.2 

81.6 

26.7% 

19.6% 

25.8% 

19.8% 

aSamplc  of  PB  students  used  to  estimate  time  savings. 

eliminations.  Three  academic  eliminations  occurred  during  the  evaluation  period,  all  in  the  BL 
condition.  Therefore,  no  analysis  of  elimination  frequencies  was  performed. 

Washbacks.  PB  and  BL  washback  frequencies  (Table  6)  were  found  not  to  be  significantly  different, 
X2(l ) = .04,  p = .84  as  was  the  comparison  of  the  composite  PLATO-using  conditions  with  the  BL  and  NP 
conditions,  XJ(  I)  = .25,  p - .62.  No  other  comparisons  were  attempted,  since  observation  frequencies  were 
low  in  the  NP  and  CP  conditions. 

Absenteeism.  Absentee  data  (Table  7)  were  analyzed  using  a two-way  classification  table.  This 
procedure  tested  for  independence  of  absenteeism  and  condition  as  the  example  in  Table  8 demonstrates. 
For  each  condition  used  in  the  classification  table,  mean  number  of  days  not  absent-for  all  four 
shreds  was  computed  to  account  for  course  length  differences.  These  variable  completion  times  were  a 
function  of  the  shred,  and  the  course  design  (Table  4).  The  results  of  the  important  condition  comparisons 
are  presented  in  Table  9.  Both  the  PB  and  NP  conditions  had  significantly  greater  absentee  frequencies 


Table  6.  Washback  Frequencies 

( Obsertat  kmsISt  udents  and  Percentage) 


1 


Condition 

Shred  Course 

A 

B 

C 

D 

Total 

BL 

2/41 

3/39 

4/58 

5/52 

14/190 

4.8% 

7.7% 

6.9% 

9.6% 

7.4% 

NP 

1/26 

— 

— 

_ 

1/26 

3.8% 

- 

- 

- 

3.8% 

CP 

- 

- 

1/34 

0/13 

1/47 

— 

- 

2.9% 

0 

2.1% 

PB 

0/25 

1/46 

0/21 

9/68 

10/160 

0 

2.2% 

0 

13.2% 

6.3% 

Table  7.  Absentee  Frequencies 

( ( fhsenut  inns  1 St  ude 

nts  and  Percentage) 

Shred  Course 

Condition 

A 

B 

C 

D 

Total 

BL 

12/41 

17/39 

18/58 

1 1/52 

58/190 

29.3%' 

43.6% 

31.0% 

21.2% 

30.5% 

NP 

21/26 

— 

— 

— 

21/26 

80.8% 

- 

- 

- 

80.8% 

CP 

- 

— 

12/34 

7/13 

19/47 

- 

- 

35.3% 

53.8% 

40.4% 

PB 

10/25 

23/46 

12/21 

35/68 

80/160 

40.0% 

50% 

57.1% 

51.5% 

50% 

Tabic  8.  Example  of  Classification  Table 


Condition 

Number  of 
Absences 

Absenteeism 

m Number  of 

Non-Absences 

Total 

BL 

58  (83.8)a 

12,906  (1 2,880.2)a 

12.964 

PB 

80  (54.2)a 

8,309  (8,334.8)a 

8.389 

138 

21,215 

21,353 

aExpcctcd  number  if  independent  results:  xa  “ 19.5,/)  < .05. 


Table  9.  Absentee  Frequency 
Comparison  Results 


PB  vs.  BL: 

X2  (1)=  19.5;  p < .05 

BL  + NP  vs. 

PB  + CP 

X1  (1)=  10.1 ; p < .05 

NP  vs.  BL: 

X2  (1)  = 15-2;  p < .05 

CP  vs.  BL: 

X2(l)=  .01; p>  .05 

k 


Wi 


when  compared  to  the  IH  frequency.  However,  the  hypothesis  that  the  BL  and  CP conditions  differed  with 
respect  to  absenteeism  was  not  rejected.  The  combined  PLATO  conditions  had  a greater  frequency  of 
absenteeism  when  compared  to  the  combined  NP  and  BL  conditions. 

In  view  of  the  relationship  between  the  evaluation  conditions  and  absenteeism,  an  ANOVA  was 
performed  on  actual  number  of  hours  absent  within  conditions.  Mean  hours  absent  and  the  ANOVA  results 
are  shown  in  Tables  10  and  1 I.  respectively.  Though  the  frequency  of  absenteeism  was  higher  for  several 
conditions  when  compared  to  the  BL  data,  the  amount  of  time  students  wcie  absent  did  not  differ 
significantly. 


Table  10.  Time  Absent 

( Ilnurs I 


Condition 

Mun 

SO 

BL 

2.45 

7.91 

NP 

5.17 

5.91 

CP 

3.61 

6.2 

PB 

2.33 

4.89 

Table  1 1.  Results  of  Absentee 
Analysis  of  Variance 


Source 

df 

MS 

F 

P 

Condition 

3 

69.89 

1.6 

.18 

Within  Cells 

404 

43.6 

Special  Individual  Assistance.  An  extended  median  test  (Siegel,  1956,  p.  179)  was  used  to  determine 
if  SI  A time  differences  were  significantly  different  between  conditions.  For  each  block,  the  results  (Table 
1 2)  showed  a trend  for  the  CP  condition  students  to  have  had  more  assistance  than  students  in  the  other 
conditions. 


Table  12.  Special  Individual  Assistance 
Extended  Median  Test  Results 


Condition  Medians 


Block 

BC 

RP 

tP 

TB 

Chi  Square  Value 

i 

.105 

.091 

.833 

.072 

X2(3)  = 236.49* 

2 

.169 

.042 

.300 

.112 

XJ  (3)  =172.61* 

3 

.183 

.196 

.389 

.152 

XJ(3)=  138.52* 

4 

.188 

.313 

.357 

.168 

X2(3)  = 1 1 2.72* 

♦ p < .001. 

Field  Fvaluation.  The  field  evaluation  resulted  in  a graduate  survey  with  the  following  sample  sizes: 
PB  = 42,  NP  = 28,  CP  = 32,  LP  = 45,  BL  = 73.  Among  these  220  graduates,  138  were  at  the  apprentice  duty 
level,  73  at  the  mechanic  duty  level,  eight  were  holding  other  assignments,  and  duty  level  information  was 
missing  for  one. 

The  first  analysis  was  performed  to  dctemiine  if  overall  field  performance  was  different  for  graduates 
of  the  five  groups.  To  account  for  variance  due  to  differences  in  duty  level,  a two-day  factorial  ANOVA  was 


21 


Table  13.  Analysis  of  Variance  of  Overall 
Field  Performance  Ratings 


Source  of  Variance 

df 

MS 

F 

Condition 

4 

.480 

1.017 

Duty  Level 

1 

2.057 

2.901* 

Condition  X Duty 

4 

.634 

.894 

Residual 

201 

.709 

*p  < .10. 


performed  with  five  levels  of  condition  and  two  levels  of  duty  (Table  13).  The  results  showed  that  there 
were  no  significant  differences  in  overall  performance  between  each  of  the  conditions,  although  duty  level 
appeared  to  be  related  to  overall  performance  as  expected.  The  overall  mean  for  graduates  in  mechanic 
duties  was  slightly,  but  not  significantly,  higher  than  the  overall  mean  for  graduates  at  the  apprentice  level. 

As  a cross-check  on  the  validity  of  the  performance  measure,  a second  analysis  was  done  by 
standardizing  ratings  on  each  of  the  33  tasks,  summing  them,  and  performing  another  ANOVA  on  this 
variable.  Specific  task  data  from  the  BL  group  could  not  be  used  in  this  analysis,  because  the  task  format 
was  not  compatible  with  that  of  the  four  groups  sampled  in  the  survey.  Consequently,  all  analyses 
concerned  with  summed  performance  of  the  33  tasks  listed  in  Appendix  A relate  only  to  conditions  PB,  LP, 
CP,  and  NP.  It  was  found  that  there  were  no  significant  performance  differences  among  graduates  of  the 
four  conditions,  F (3,  145)=  1.271,  p > .25.  To  determine  if  certain  tasks  were  related  to  overall 
performance  and  if  those  specific  tasks  were  differentially  influenced  by  PLATO  and  non-PLATO  training, 
two  additional  analyses  were  performed.  The  first  of  these  analyses  involved  a multiple  regression  design,  in 
which  the  dependent  variable  was  the  overall  field  performance  rating  of  students  at  the  apprentice  level, 
and  the  independent  variables  were  12  tasks  which  had  the  highest  rating  variances.  Four  of  these  tasks 
were  significantly  related  to  the  overall  rating.  The  four  tasks  were  of  significantly  different  difficulty.  The 
task  dealing  with  tools  and  safety  received  the  highest  mean  rating  of  all  tasks  (2.9)  indicating  it  was 
peihaps  the  easiest  to  perfomi.  The  remaining  three  tasks  received  approximately  the  same  mean  score, 
2.235,  2.285,  2.229,  for  technical  orders,  data  collection  forms,  and  power  steering,  respectively,  indicating 
approximately  equal  difficulty.  These  four  tasks  produced  a multiple  R of  .743  and  are  presented  in  Table 
14.  The  second  analysis  was  a 4 X 4 ANOVA  (four  levels  of  tasks  and  four  levels  of  conditions).  Again,  it 
was  found  that  condition  was  not  a factor  in  the  ability  of  a student  to  perform  these  tasks. 


Table  14.  Tasks  Related  to  Overall  Performance  Rating  of  Special 
Purpose  Vehicle  Repairman  Training  Graduates 


Ta»k 

Mean  Performance 
Rating 

Apply  safety  practices  with  tools  and 

equipment 

2.97 

Use  technical  orders 

2.17 

Use  maintenance  data  collection  of  forms 

2.28 

Power  steering  system 

2.23 

Discussion  of  Course  effectiveness.  Almost  all  the  common  course  segment  instructional  effectiveness 
indicators  showed  the  ISD  versions  to  be  more  instructionally  efficient.  Moreover,  PLATO  course  versions 
were  at  least  as  effective  as  versions  employing  alternative  media.  The  block  examination  findings  showed 
the  PB  condition  students'  performance  to  be  satisfactory-in  a purely  numerical  sense,  even  better  than  the 
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other  conditions.  This  latter  case  could  be  the  result  of  lower  test  difficulty  and  a policy  of  teaching 
towards  the  test  items.  Student  eliminations,  washbacks,  and  S1A  time  differences  between  the  BL  and  PB 
conditions  were  not  significant.  The  higher  absentee  frequency  for  the  PB  students  could  have  been  a 
function  of  factors  not  related  to  the  PB  instructional  environment  since  the  same  increase  was  noted  for 
other  concurrent  conditions  (NP,  CP).  Field  evaluation  findings  also  provided  evidence  for  the  contention 
that  the  PB  condition  was  at  least  as  effective.  Field  supervisors  did  not  report  an  appreciable  change  in  the 
quality  of  the  courses’  graduates.  In  fact,  this  graduate  cvauation  study  was  unusual  in  that  every  item  was 
at  least  satisfactory.  More  often,  graduate  evaluation  shows  at  least  one  and  typically  several  deficiencies.  It 
seemed  evident  that  the  new  version  was  instructionally  effective,  and  because  of  the  28  percent  time 
reduction,  it  was  considered  as  instructionally  more  efficient  than  the  preceding  versions. 

It  can  be  concluded  that  both  PLATO-using  versions  were  effective.  In  addition  to  the  previous  line 
of  reasoning  for  the  PB  condition,  the  CP  condition  block  examination  scores  were  not  significantly 
different  from  the  NP  and  BL  condition  scores.  This  finding  was  somewhat  confounded  by  the  fact  that  CP 
students  had  the  greatest  amount  of  additional  assistance.  Interviews  with  instructor  personnel  indicated 
that  the  CP  instructors  were  concerned  about  the  unvalidated  PLATO  lessons’  effectiveness.  Also,  due  to 
the  depth  of  instruction  changes,  instructors  found  it  necessary  to  review  and  present  additional  material 
after  their  students  had  interacted  with  PLATO.  In  most  areas,  this  occurred  immediately  during  the 
normal  6-hour  day.  But  it  was  periodically  necessary  to  use  SIA  time  for  this  augmentation.  The  impact  on 
the  block  examination  scores  could  only  be  surmised;  yet,  this  increased  SIA  time  was  not  considered  to  be 
substantial.  Also,  the  additional  assistance  might  not  have  been  necessary,  since  review  of  the  PLATO 
lessons  was  probably  a function  of  initial  concern  for  their  effectiveness.  Other  indices  of  system 
effectiveness  showed  the  CP  condition  to  be  at  least  as  effective  as  the  BL  and  NP  conditions. 

All  in  all,  the  ISD  version  was  instructionally  efficient,  and  PLATO  was  an  integral  and  effective 
component  of  that  instructional  system.  Similarly,  PLATO  worked  well  in  the  course  before  the  changes, 
through  the  lessons  were  not  really  designed  for  use  there. 

PLATO  System  Effectiveness 

The  most  perplexing  problem  faced  by  the  evaluation  team  was  the  creation  of  a design  to  determine 
if  PLATO  IV-as  used  by  the  project  personnel  at  Chanute-was  more  or  less  instructionally  efficient  than 
other  presently  available  media.  To  determine  this  relative  media  efficiency  without  developing  numerous 
alternative  lessons  was  the  basic  constraint.  Thus,  the  problem  was  to  ascertain  if  lesson  achievement  and 
completion  time  differences  that  existed  between  conditions  were  related  to  a class  of  medium;  i.e.,  CAI  or 
non-CAI.  The  ideal  design  would  control  for  the  effects  of  ISD  influences,  subject  matter  differences,  and 
the  nature  of  the  medium.  ISD  influences  were  chiefly  changes  to  the  scope  and  depth-of-instruction,  and 
lesson  strategies.  Subject  matter  characteristics  (e.g.,  lesson  complexity)  directly  influenced  student 
performance.  Also,  since  PLATO  was  an  individualized  medium,  efficiency  comparison  had  to  be  made 
within  that  general  class  of  medium.  The  results  might  clearly  indicate  that  within  the  context  of  PLATO’s 
application  at  Chanute,  PLATO  lesson  efficiency  was  less  than,  greater  than,  or  about  the  same  as  other 
media  of  a similar  nature. 

No  single  design  within  the  constraints  laid  on  the  evaluation  team  could  simultaneously  control  for 
the  key  variables.  Hence,  two  methods  were  used  to  examine  the  relative  efficiency  of  PLATO.  First,  a 
topical  test  was  employed  in  determining  achievement  differences  between  conditions  while  controlling  for 
some  ISD  procedural  influences.  Second,  a small  scale  programmed  text  versus  PLATO  lesson  comparison 
controlled  for  subject  matter,  lesson  design,  and  nature  of  media  differences. 

Procedures.  A 50-item  topical  test  was  prepared  to  assess  achievement  differences  between  each 
condition  (excepting  BL).  Development  constraints  precluded  trying  out  and  revising  the  instrument.  The 
test  reliability  was  low,  consequently  the  methods  and  results  will  not  be  included  in  this  report. 

Determining  whether  time  savings  were  attributable  to  PLATO  required  comparisons  with  other 
self-paced  media;  therefore,  four  programmed  texts  were  prepared  to  replace  four  PLATO  lessons.  The 
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lessons  were  selected  from  those  lessons  which  used  computer  graphics  (rather  than  microfiche  slides)  to 
present  illustrations,  so  that  all  screen  displays  could  be  electrostatically  copied  using  a printer  designed  for 
the  PLATO  system.  The  off-line  versions  paralleled  the  on-line  wherever  possible,  the  major  differences 
being:  (a)  response  feedback  was  provided  at  the  top  of  the  next  page,  (b)  feedback  was  not  as  varied,  (c) 
no  animations  were  possible,  and  (d)  any  departure  from  the  programmed  sequence  was  at  the  discretion  of 
the  student.  Between  40  and  70  students  received  either  version  of  the  lesson  during  the  last  4 months  of 
1975.  Lesson  completion  times,  end-of-lcsson  test  scores,  and  student  responses  to  two  attitude  items  were 
collected.  Attitude  items  asked  the  student  for  his  preference  for  PLATO  as  opposed  to  programmed  texts, 
and  asked  which  medium  he  found  easier  to  use.  Attitude  data  were  not  collected  from  the  PLATO  lesson 
control  group. 

Results.  Descriptive  statistics  for  the  measures  taken  to  compare  the  PLATO  and  programmed  text 
lesson  versions  are  reported  in  Table  15.  /'-tests  were  performed  on  tire  score  means,  the  results  being  not 
significant  for  any  lesson  pair.  It  should  be  noted  that  each  PLATO  version  of  the  lesson  had  a higher  mean 
score,  the  probability  of  this  event  being  p = .0625. 


Table  15.  Performance  Data  for  PLATO  and  Programmed  Text  Comparison 


Lesson 

Typ« 

Achievement 
(%  Correct) 

Test 

Failure 

Time 

(Minutes) 

N 

M Lesson 

Test  Score 

SD 

M 

SD 

Cooling 

PLATO 

70 

95.9 

13.2 

5 

40.8 

15.3 

System 

PT 

44 

93.8 

14.8 

7 

32.3 

10.9 

Warning 

PLATO 

58 

91.6 

11.5 

7 

13.7 

7.9 

Systems 

PT 

41 

87.3 

18.0 

9 

14.4 

4.1 

Clutches 

PLATO 

56 

91.2 

15.8 

8 

24.5 

12.4 

PT 

42 

88.0 

15.9 

6 

26.0 

10.8 

Brakes 

PLATO 

51 

95.8 

7.5 

0 

46.0 

18.5 

PT 

46 

93.0 

13.1 

4 

41.7 

11.2 

Lesson  completion  times  were  analyzed  in  a similar  fashion  as  lesson  scores.  The  cooling  system 
lesson  pair  was  found  to  have  significantly  different  completion  times,  f(112)  = 3.46,  p<  .01.  No  other 
comparison  attained  significance. 

A four-way  classification  table  was  used  to  determine  if  lesson  failure  was  independent  of  media.  The 
results  indicated  that  programmed  texts  had  a significantly  higher  failure  rate,  x2  (1 ) = 3.6,  p > .05. 

Attitude  item  response  frequencies  are  depicted  in  Table  16.  Most  students  preferred  PLATO  to 
programmed  tests,  and  found  the  CA1  lessons  easier  to  use. 

In  summary,  the  results  of  the  comparison  of  PLATO  lessons  with  a comparable  lesson  version  using 
the  programmed  text  medium  found  little  difference  in  terms  of  lesson  completion  times  and  lesson  text 
scores.  Programmed  text  versions  had  a substantially  greater  failure  rate  and  were  less  desirable  to  students. 

Discussion  of  Comparison  Findings.  The  results  of  comparing  PLATO  lessons  with  an  off-line 
programmed  text  version  demonstrated  that  the  PLATO  lessons  were  more  instructional^  effective: 
end-of-lcsson  test  scores  were  consistently  higher,  and  fewer  failures  were  observed.  The  higher  programmed 
text  failure  rate  may  have  been  the  result  of  the  loss  of  the  forced  review  feature-students  were  required  to 
repeat  a lesson  segment  if  they  did  not  reach  the  author’s  standards  for  within-lesson  testing-when  the 
lessons  were  adapted  for  off-line  use.  Often,  the  fact  that  students  did  not  review  a portion  of  a 
programmed  text  when  they  made  mistakes  may  have  been  a factor.  Also,  many  students  tended  to  look  at 
answers  on  the  next  page  of  a programmed  text  before  answering  the  questions.  It  is  likely  that  the  higher 
failure  rate  was  attributable  to  a combination  of  these  two  circumstances. 
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Table  /z>  Altitude  Item  Response  Frequencies 
for  PLATO-Programmed  Text  Comparison 


□ •finitely 

Dujgree  Uwagree 

Undecided 

Agree 

Definitely 

Agree 

No 

Litton 

1 2 

3 

4 

S 

Retponte 

Item  1 1 like  programmed  texts  more  than  Pi  A TO. 

Cooling  System 

10  9 

13 

3 

3 

Warning  System 

21  ‘1 

5 

3 

3 

Clutches 

l‘>  1 1 

5 

3 

3 

i 

Brakes 

IK  9 

11 

*> 

6 

Total 

74  28 

34 

11 

15 

i 

M = 2.16;  Si)  = 

1.2*);  N * 1 73 

Item  2 PI  ATO  is  easier  to  work  w ith  than  programmed  texts 

Cooling  System 

4 5 

6 

13 

16 

Warning  System 

5 4 

6 

9 

17 

Clutches 

3 1 

9 

10 

19 

Brakes 

9 2 

6 

14 

15 

Total 

21  12 

27 

46 

67 

M = 3.73;  Sl>  = 

1.36;  N - 1 73. 

If  the  effort  had  been  made  to  validate  the  off-line  lesson  versions  prior  to  data  collection,  the 
resultant  programmed  texts  would  probably  have  required  more  time  to  complete.  Basically,  the  lessons 
were  technically  accurate  and  the  presentation  of  material  by  the  instructional  strategies  employed  was 
acceptable.  Hence,  revision  would  be  of  the  form  most  commonly  used  in  this  situation;  additional 
within-lesson  practice  in  areas  related  to  the  end-of-lesson  lest.  Additional  practice  would  undoubtedly  have 
brought  about  a corresponding  increase  in  lesson  time.  Thus,  the  net  effect  would  be  the  same  as  was 
indicated  earlier;  the  programmed  text  versions  would  require  more  time,  making  them  less  instructionally 
efficient  than  the  PLATO  versions. 

As  it  was.  the  intention  of  this  comparison  to  investigate  the  effectiveness  of  the  Chanute  author’s 
PLATO  lessons  relative  to  an  off-line  version  of  the  same  lessons,  the  results  should  not  be  considered  as 
generalizable  outside  of  that  environment.  It  is  possible  that  given  a different  target  population  and  subject 
matter  area,  programmed  texts  may  be  more  efficient.  Even  generalizing  to  the  entire  vehicle  maintenance 
lesson  population  requires  qualification.  The  lessons  were  selected  from  those  which  did  not  use  the 
microfiche  capability  and  a case  could  be  made  that  the  microfiche  use  was  related  to  the  more 
sophisticated  lessons.  For  example,  one  lesson  used  microfiche  in  a randomized  matching  routine,  which 
required  students  to  identify  ignition  system  components  without  error  before  progressing.  Though  not 
probable,  it  may  be  that  lessons  like  this  one  would  not  be  as  instructionally  efficient  as  a carefully 
prepared  programmed  text.  All  in  all,  the  Chanutc  PLATO  lessons  were  somewhat  more  efficient  and 
preferred  by  students. 

It  may  be  possible  to  extrapolate  cost  savings  from  these  tenuous  indications;  however,  the  merit  of 
such  an  activity  might  be  questioned.  It  will  be  e .dent  from  the  cost  analysis  that  potential  savings  had  to 
be  sizable  for  Chanute’s  style  of  CAI  lesson  to  be  cost-effective.  To  achieve  a cost-effective  operation  with 
lessons  which  arc  slightly  more  efficient  requires  large  numbers  of  lessons  and  a high  degree  of  student  use. 
Typically  this  situation  is  not  possible  in  a military  training  environment. 
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PLATO  Couiseware 

Procedures.  Evaluation  resources  and  circumstances  rendered  comparisons  at  the  lesson  level 
impractical;  consequently,  absolute  measures  of  lesson  effectiveness  were  desirable;  i.e..  each  lesson  should 
be  assessed  on  the  basis  of  goal  attainment.  The  goal  in  this  case  was  the  lesson  validation  criterion 
employed  by  the  ISD  team.  A lesson  was  said  to  be  validated  when  90  percent  of  a 30-student  sample 
achieved  the  stated  standard  on  their  first  attempt  at  the  lesson.  Once  the  validation  criterion  was  met, 
lesson  improvement  activities  were  no  longer  mandatory.  The  validation  data  collection  was  retained  Juring 
a lesson’s  post-validation  period  for  a longitudinal  study  of  goal  attainment.  Student  scores,  completion 
times,  and  number  of  lesson  repeats  were  collected  on  each  lesson,  so  that  data  collection  lasted  until 
February  1976  for  some  of  the  latest  validating  lessons. 

Results.  The  primary  indicators  of  lesson  performance  are  presented  in  Table  17.  The  10  percent 
maximum  allowable  failure  rate  selected  by  the  ISD  team  for  lesson  validation  purposes  was  used  as  a basis 
to  judge  the  longitudinal  performance  of  a lesson-  If  the  failure  rate  was  less  than  10  percent  after  the 
lesson  achieved  its  validated  status,  it  was  considered  instructionally  stable,  requiring  only  periodic 
monitoring  of  the  performance  indicators  to  insure  that  changes  in  the  target  population  or  instructional 
setting  did  not  decrease  its  effectiveness.  If  the  failure  rate  was  greater  than  15  percent,  performance  data 
for  the  40  students  who  received  the  lesson  prior  to  20  January  1976  were  reviewed  and  the  updated  rate 
cited  in  the  comments  section.  A failure  rate  falling  in  and/or  remaining  in  the  10-  to  20-percent  range 
indicated  that  the  lesson  should  be  monitored,  then  revised  if  deemed  necessary  by  the  course 
administration.  Revision  was  considered  mandatory  with  a failure  rate  greater  than  20  percent.  Of  the  35 
instructional  units  prepared,  four  were  in  need  of  revision  and  eight  required  monitoring  as  of  January 
1976. 

The  self-paced  nature  of  the  CAI  courseware  could  result  in  additional  time  savings  if  the  group-paced 
Special  Purpose  Vehicle  Repairman  course  went  to  a self-paced  design.  An  estimate  of  additional  common 
course  segment  time  savings  can  be  found  by  subtracting  mean  lesson  time  t20  hours)  and  estimates  of 
block  examination  time  (3  hours)  and  on-line  remedial  instruction  ( I hour)  from  the  total  on-line  time  for 
the  four  blocks  (30  hours).  The  residual  6 hours  is  the  mean  time  that  students  spent  on  enrichment 
material  over  the  four  common  blocks  while  waiting  for  the  slower  students  to  complete  lessons.  The 
common  segment  could  be  shortened  by  this  amount  if  the  loss  of  enrichment  material  did  not  impair 
attainment  of  objectives. 

Discussion  of  Lesson  Performance  Results.  The  courseware  performance  indices  signified  that  the 
lessons,  as  a group,  were  instructionally  effective.  With  some  additional  revision,  it  is  apparent  that  failure 
rates  can  be  kept  at  approximately  10  percent  per  lesson.  From  a longitudinal  standpoint,  the  PLATO 
courseware  retained  its  effectiveness  despite  changes  in  the  target  population.  The  entering  ability  of  the  PB 
condition  students  increased  over  the  12-month  data  collection  period.  Higher  ability  students  should 
perform  well  on  the  end-of-lesson  tests.  However,  it  is  unknown  whether  these  lessons  would  have  retained 
their  effectiveness  had  the  entering  ability  of  the  target  population  declined.  It  is  possible  that  increase  in 
ability  may  be  related  to  an  attitudinal  decline.  Later  PB  students  did  not  like  PLATO  lessons  as  well  as 
early  students,  and  one  explanation  is  that  the  higher  ability  students  found  the  lessons  to  be  too  easy. 
Though  the  longitudinal  data  indicated  courseware  effectiveness  to  be  stable,  there  is  reason  to  believe  that 
changes  in  student  entering  ability  may  have  produced  undesirable  changes  in  performance  and  attitude. 

A further  qualification  upon  the  interpretation  of  student  performance  arises  from  a particularly 
frustrating  student  behavior.  A number  of  students  attempted  to  “beat  the  system”  by  finding  ways  of 
circumventing  lessons.  A favorite  tactic  was  to  page  quickly  through  the  lesson,  learning  just  enough  to 
satisfactorily  respond  to  the  periodic  interaction.  Then,  upon  taking  the  end-of-lcssr-n  test,  the  student 
familiarized  himself  with  the  items  so  that  he  could  key  upon  the  subject  matter  related  to  these  items. 
Such  a tactic  arbitrarily  inflated  failure  rates. 

PLATO  courseware  is  discussed  extensively  in  the  instructional  materials  development  section.  Many 
of  the  topics  pertain  to  the  effectiveness  of  these  lessons;  consequently,  the  interested  reader  is  referred  to 
this  section. 
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Table  1 7.  PLATO  Courseware  Performance  Data 


Instructional 

Unit 

Tit  It 

Mean 

Validation  Lesson 
Date  Time 

Failure 
Rata  (%) 
(Before 
Octobar) 

N 

Comments 

103 

Gasoline  Engine  Principles 

lOJun  30 

11 

63 

Recpmmend  Monitoring 
Student  Performance 

104a 

Identification  of  Engine  Parts 

14  Apr 

34 

6 

114 

104b 

Identification  of  Engine  Parts 

14  Apr 

34 

14 

113 

Recommend  Monitoring 
Student  Performance 

105 

Cooling  System 

14  Apr 

44 

12 

102 

Recommend  Monitoring 
Student  Performance 

106 

Engine  Lubrication  System 

19  Jun 

15 

18 

33 

Recommend  Monitoring 
Student  Performance 

201a 

Air  and  Exhaust  System 

28  May 

20 

10 

99 

201b 

Air  and  Exhaust  System 

23  May 

38 

28 

109 

22%  Last  40  Cases  (20  Jan  76) 
Revision  Recommended 

202a 

Electrical  Fundamentals 

18  Aug 

148 

18 

33 

Recommend  Monitoring 
Student  Performance 

202b 

Batteries 

28  May 

15 

2 

99 

203a 

Electrical  Schematics 

28  May 

28 

3 

33 

203b 

Electrical  Schematics 

13  Jun 

28 

6 

33 

203c 

Electrical  Schematics 

18  Aug 

28 

9 

33 

204 

Starters 

18  Aug 

48 

6 

33 

205a 

DC  Generators  and 

Charging  System 

15  Jan  76 

60 

21 

33 

Recommend  Revision 

205b 

AC  Charging  System 

1 5 Jan  76 

37 

18 

33 

Recommend  Monitoring 
Student  Performance 

206a 

Battery  Ignition  System 

13  Jun 

43 

18 

90 

7%  Last  40  Cases  (20  Jan  75) 

206b 

Battery  Ignition  System 

25  Jun 

23 

18 

65 

5%  Last  40  Cases  (20  Jan  75) 

206c 

Battery  Ignition  System 

11  Apr 

51 

5 

1 18 

207 

Emission  System 

1 5 Aug 

32 

9 

33 

301 

Diesel  Engines 

25  Jun 

31 

21 

109 

12%  Last  40  Cases 

Monitoring  Performance 

R ec  om  mended 

303 

Lighting  System 

2 May 

13 

10 

109 

304 

Vehicle  Warning  System 

25  Jun 

11 

18 

65 

10%  Last  40  Cases  (20  Jan  75) 

305 

Clutches 

18  May 

25 

4 

109 

307 

Principlesof  Hydraulics 

14  Apr 

32 

19 

130 

10%  Lait  40  Cases  (20  Jan  75) 

308 

Automatic  Transmissions 

18  May 

48 

37 

109 

35%  Last  40  Cases  (20  Jan  75) 
Recommend  Revision 

401 

Propeller  Shaft 

17  Apr 

20 

17 

142 

5%  Last  40  Cases  (20  Jan  75) 

402 

Differentials 

8 Jul 

78 

21 

65 

10%  Last  40  Cases  (20  Jan  75) 

403 

Transfer  Cases  and  Power 
Take-Offs 

30  Jun 

10 

21 

65 

1 5%  Last  40  Cases  (20  Jan  75) 
Recommend  Monitoring 
Student  Performance 

404 

Suspension  Systems 

2 Sep 

13 

0 

33 

405a 

Power  Assisted  Brakes 

26  Aug 

42 

0 

33 

405b 

Power  Assisted  Brakes 

26  Aug 

10 

9 

33 

405c 

Power  Assisted  Brakes 

26  Aug 

25 

6 

33 

405d 

Power  Assisted  Brakes 

2 Sep 

29 

27 

33 

Revision  Recommended 

406 

Mechanical  Steering 

30  Jun 

32 

5 

65 

407 

Power  Steering 

2 Sep 

26 

12 

33 

Recommend  Monitoring 
Student  Performance 

Instructional  Effectiveness  Conclusions 

The  instructional  effectiveness  of  the  Special  Purpose  Vehicle  Repairman  course  and  its  principal 
instructional  medium,  PLATO  were  assessed.  The  procedure  was  to  compare  key  instructional  effectiveness 
indices  to  determine  the  relative  efficiency  of  the  new  course  and  the  PLATO  medium.  Like  most 
evaluative  field  research,  site  conditions  distorted  somewhat  the  clarity  of  individual  findings.  However,  as 
in  the  situation  where  one  views  a newspaper  photograph  from  a distance  and  sees  a meaningful  picture 
rather  than  a collection  of  dots,  the  individual  findings  do  portray  the  effectiveness  issue  when  viewed  as  a 
composite. 

Course  performance  indices  showed  the  new  Special  Purpose  Vehicle  Repairman  courses  to  be  highly 
efficient.  Data  analysis  revealed  no  major  group  differences  between  the  evaluation  conditions  except  for 
reduced  student  completion  time  for  the  PB  condition.  The  28  percent  time  reduction  in  the  common 
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course  segment  was  attributable  to  the  use  of  individualized,  self-paced  media,  group-paced  instructional 
design,  and  adherence  to  the  ISD  philosophy  of  preparing  lean,  but  effective  and  relevant,  instruction. 

Student  achievement  in  the  new  course  was  similar  to  that  of  prior  versions  with  and  without  the  use 
of  PLATO.  Consequently,  PLATO  lessons  can  be  said  to  have  produced  comparable  learning  outcomes.  It  is 
evident  that  the  Chanute  PLATO  authors  were  able  to  develop  effective  lessons  for  presenting  factual  and 
conceptual  information.  Moreover,  there  was  some  indication  that  these  CAI  lessons  may  have  been  more 
efficient  than  a programmed  text  counterpart.  Student  performance  did  not  fluctuate  due  to  instructor 
differences  because  of  the  standardized  characteristic  of  the  CAI  lessons.  PLATO  (as  a technical  training 
tool)  and  the  consolidated  ISD  and  PLATO  development  team  lessons  were  instructional  effective. 

The  quantitative  evaluation  findings  supported  the  observations  of  instructor,  project,  and  evaluation 
team  personnel.  Another  way  of  conveying  the  point  that  PLATO  and  the  new  course  were  performing 
their  role  is  to  compare  student  behavior  in  different  environments.  Instructors  in  the  PLATO  laboratory 
seldom  cautioned  students  to  remain  alert,  even  if  their  students  had  experienced  the  majority  of  their 
PLATO  lessons.  Instructors,  management,  and  evaluation  team  personnel  had  noted  that  this  was  not  the 
case  when  student— in  some  situations  even  the  same  students-werc  interacting  with  programmed  texts  and 
workbooks.  This  type  of  CAI  system  captured  and  retained  student  attention  for  the  same  reasons  that 
laboratory  tasks  do  so.  The  student  interacted  with  and  manipulated  his  learning  environment.  Since  this 
situation  was  more  prevalent  in  the  new  course  with  its  PLATO  lessons  and  its  greater  emphasis  on 
laboratory  activity,  it  is  reasonable  to  propose  that  this  student  activity  had  a direct  bearing  on  the 
efficiency  of  instruction. 


IV.  COST  FACTORS 

Introduction 

This  section  examines  the  costs  and  benefits  associated  with  the  PLATO  systems  as  used  at  Chanute. 
Since  no  competing  system  which  had  the  same  goals  and  outcomes  was  available  for  comparison  purposes, 
a true  cost-effectiveness  study  was  not  possible.  Where  possible,  comparative  information  will  be  given.  For 
example,  when  discussing  costs,  the  developmental  and  operational  expenditures  can  be  compared  with  a 
similar  system  which  would  not  rely  on  PLATO.  Moreover,  PLATO-related  benefits  will  be  presented  as 
actual  and  potential  savings,  and  those  tangible  benefits  upon  which  it  is  difficult  to  place  a dollar  value: 
student  attitudes,  improved  methods,  and  enhanced  safety. 

Method 

PLATO-related  cost  and  benefit  data  were  collected  during  the  first  9 months  of  1976.  Collection 
procedures  primarily  involved  interviews  with  personnel  directly  associated  with  the  design,  development, 
and  operation  of  the  PLATO  instructional  program.  Applicable  records  and  reports,  available  from  CERL 
and  Chanute  activities,  were  also  used  as  data  sources.  Cost  figure  details  varied  considerably,  because  a 
breakdown  of  costs  was  not  kept.  The  level  of  detail  was  influenced  by  the  conditions  of  the  service  test. 
For  example,  the  attempt  to  appraise  the  total  lesson  development  cost  was  heavily  contaminated  by  the 
fact  that  much  of  the  material  developed  for  the  original  target  course  was  modified  for  the  Special  Purpose 
Vehicle  Repairman  course.  As  a result,  cost  comparisons  between  course  versions  must  be  on  a very  global 
level. 

Certain  assumptions  are  necessary  to  provide  comparative  information.  It  will  be  assumed  that  the 
Special  Purpose  Vehicle  Repairman  course  without  PLATO  would  have  used  programmed  texts,  where  CAI 
lessons  were  used  and  conventional  paper-and-pencil  block  examinations  instead  of  the  computer- 
administered  tests. 

Results  and  Discussion 

Equipment  Costs.  Table  18  lists  equipment  costs  during  the  service  test.  The  Varian  printer  was  a 

28 


I 


Table  IS.  Capital  Cost  of  Equipment 


Device 

Number 

Cost 

Terminal 

30 

$156,150 

Touch  panel 

30 

16,320 

Varian  Printer 

1 

7.588 

Audio  Unit 

1 

2.000 

Microwave  System 

1 

9,630 

Total  Capital  Investment 

$191,688 

useful  device  which  made  an  electrostatic  copy  of  a PLATO  panel  display.  The  audio  unit  was  never  used  in 
conjunction  witht  he  project,  since  only  one  was  available  and  the  device  had  poor  reliability. 

Facilities  Preparation  Costs.  Electrical  outlets,  and  pneumatic  lines  and  controls  were  installed  in  the 
PLATO  terminal  room.  Power  and  air  outlets  were  placed  near  each  of  the  .10  carrels  in  which  the  terminals 
were  placed.  Each  carrel  had  a variable-intensity  light  for  note-taking  purposes.  Costs  associated  with 
facilities  preparation  arc  presented  in  Table  19. 


Table  / V.  Facilities  Preparation  Costs 


Facilities 

Cost 

Electrical  and  Pcneumatic 

$1,822.50 

Carrels  (3) 

2.700.00 

Variable-intensity  light  with  anti-theft  cable 

280.00 

Total 

S4.802.50 

Operating  Costs.  To  maintain  the  operational  course  involved  some  yearly  expenses  directly 
attributed  to  PLATO.  These  expenses  were  classified  as  computer  leasing,  terminal  maintenance, 
communications,  facilities,  and  support  personnel.  Table  20  depicts  these  costs  fot  the  operational  SPVR 
course.  If  PLATO  had  not  been  used  in  this  course,  recurring  costs  associated  with  programmed  texts  and 
block  examinations  would  have  been  present.  Ordering,  storage,  and  handling  of  replacement  programmed 
texts,  tests,  and  answer  sheet  forms  would  have  cost  approximately  $2,500  per  year.  Thus,  with  the 
30-terminal  system  employed  at  Chanute.  the  operating  expenses  were  approximately  $87,500  higher  than 
a similar  system  which  would  not  use  PLATO. 


Table  20.  Operational  Costs 


Item 

Annual  Expense 

Computer  leasing 

$45,000 

(1 ,500/tcrminal) 

Communications 

6,120 

(204/terminal) 

terminal  maintenance 

30,000 

(1 ,000/terminal) 

Facilities'* 

1,318 

Utilities11 

295 

Support  personnel1 

7,200 

(Estimate) 

Total 

$89,933 

“Prorated  costs  of  utilities  (air  conditioning,  lighting,  compressed 
air,  etc.)  and  maintenance  for  the  PLATO  lab  area,  1,600  sq.  it. 

^ Power  costs  to  operate  terminals  based  on  a mean  usage  of  1,092 
hours  per  year  per  terminal,  and  S00  watts  per  terminal. 

c Based  on  two  authois  (GS-9  and  GS  S ratings)  and  a computer 
coder  (GS  7 spending  about  20%  of  their  time  on  operational 
maintenance  tasks. 

Developmental  Costs  From  the  point  where  the  ISD  and  PLATO  projects  merged,  it  cost  $52,200  to 
adapt  lessons  developed  for  the  General  Purpose  Vehicle  Repairman  course,  create  new  courseware, 
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program  CMI  routines,  and  develop  14  block  examinations.  The  cost  figures  were  based  on  6,517  civilian 
and  military  man-hours.  With  such  gross  figures  available,  no  attempt  was  made  to  use  this  information  for 
comparative  development  costs.  Yet,  the  following  estimate  of  comparative  costs  was  made.  For  the  simple 
tutorial  lesson  produced  in  a quasi-team  approach  (see  Instructional  Materials  Development  Section), 
approximately  100  man-hours  were  required  to  develop  one  student  contact  hour  on  PLATO.  It  takes 
approximately  40  hours  to  develop  one  draft  programmed  text  contact  hour.  The  CA1  lesson,  on  the  other 
hand,  was  ready  for  validation  tryouts.  Using  an  estimated  hourly  wage  of  $7.80  which  includes  indirect 
support  costs,  the  cost  for  a tryout-ready  CAI  lesson  was  about  $780  dollars  per  contact  hour.  Using  the 
same  $7.80  figure  and  adding  in  typing  and  printing  expenses,  the  tryout-ready  programmed  texts  cost 
about  $470.00  per  contact  hour,  a difference  of  $310.00.  Obtaining  the  validated  lesson  would  narrow  this 
price  difference  somewhat  since  a programmed  text  would  generally  be  more  expensive  to  revise.  Lesson 
revision  typically  involved  adding  questions,  improving  text,  and  eliminating  technical  and  grammatical 
errors.  These  activities  could  be  accomplished  more  easily  on  PLATO  because  of  edit  capability.  With 
programmed  text,  changes  involved  retyping  pages,  cutting  and  pasting,  and  reproducing  enough  copies  for 
the  next  tryout.  Moreover,  obtaining  a final  version  of  a programmed  text  involved  retyping  the  entire  text, 
performing  artwork,  and  proofreading.  After  a PLATO  lesson  was  validated,  it  was  ready  immediately  for 
use.  Our  best  estimate  was  that  overall  the  price  difference  was  less  than  $155  per  contact  hour.  ($310 
difference  minus  $155  typing  and  printing  costs.) 

For  each  block  examination,  course  authors  used  the  same  computer  coding  routine;  hence,  the  basic 
differences  in  development  costs  were  small.  After  the  routine  was  duplicated  into  a lesson  file,  a clerk 
typed  in  the  questions.  No  reproduction  costs  were  therefore  incurred.  The  original  routine  cost 
approximately  $500.00.  It  is  assumed  that  reproduction,  assembly,  and  handling  charges  would  balance  this 
development  cost,  since  eventually  60  block  examinations  were  put  on-line.  Consequently , cost  differences 
were  small. 

The  cost  to  develop  the  routine  which  automatically  performed  the  test  analyses  required  by  ATC 
(see  Management  section)  could  not  be  determined.  The  benefits  in  temis  of  real  savings  and  aesthetic 
consideration  were  great  and  will  be  discussed  later. 

It  was  apparent  that  the  PLATO  version  of  the  Special  Purpose  Vehicle  Repairman  course  cost  more 
to  develop.  Using  a difference  of  $1  55  to  develop  a student  contact  hour  and  a mean  number  of  contact 
hours  of  20,  the  difference  for  the  lessons  alone  was  $3,100.  For  the  additional  routines  of  a CMI  nature, 
one  might  add  several  thousand  dollars. 

Cost  Savings 

Real  Savings.  In  the  military  environment,  shortening  training  time  produces  real  savings.  For 
example,  the  reduction  of  the  common  course  segment  by  28  percent  brought  about  a yearly  savings  of 
$180,000.  This  saving  was  attributable  to  all  the  changes  brought  about  by  the  redesign  of  the  courses. 
What  portion  of  this  time  savings  was  attributable  to  PLATO  is  unknown,  but  an  estimate  can  be  provided. 
From  Army  experiences  at  Aberdeen  Proving  Grounds  (Department  of  the  Army,  1975)  and  the  results 
from  the  programmed  text/PLATO  lesson  comparison,  it  was  reasonable  to  assume  that  the  CAI  lessons 
provided  a time  savings.  The  Aberdeen  report  suggests  at  least  a 10  percent  time  reduction.  The 
programmed  text-CAl  lesson  comparison  suggested  that  the  off-line  version  would  require  additional 
practice  or  explanation  in  order  that  the  failure  rates  would  not  be  different.  From  these  experiences,  it  was 
assumed  that  Chanute  PLATO  lessons  saved  approximately  10  percent  over  programmed  text  equivalents. 
The  mean  on-line  time  for  mainstream  lessons  was  20  hours.  If  programmed  texts  were  used,  it  would  have 
increased  training  time  by  2 hours  per  student. 

Block  examination  administration  and  critiquing  time  was  cut  in  half  by  the  use  of  the  PLATO 
system.  A net  saving  of  6 hours  was  experienced  for  the  four  common  course  segment  blocks.  Therefore,  an 
estimated  time  savings  attributed  to  the  Chanute  use  of  PLATO  IV  was  about  8 hours  or  one  and  one-third 
training  days.  At  $48  per  student  per  day  (provided  by  Chanute  AFB  Requirements  Section)  and  a total 
student  flow  of  375  per  year  for  the  four  Special  Purpose  Vehicle  Repairman  courses,  the  estimated  savings 
was  approximately  $23,000. 
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If  it  can  be  assumed  that  time  savings  for  the  instructor’s  tasks  allowed  him  to  be  productive  in  other 
areas,  then  instructor  time  savings  produced  real  savings.  For  example,  if  instructors  no  longer  had  to  grade 
end-of-lesson  tests  and  block  examinations,  and  they  had  more  time  available  because  test  analyses  were 
automatically  performed,  then  they  could  use  that  additional  time  to  revise  test  items.  Thus,  instructor 
productivity  could  increase,  and  a method  of  appraising  the  dollar  value  of  this  productivity  could  be 
found.  However,  because  instructor  time  savings  were  typically  spread  out  over  a long  period  of  time  rather 
than  grouped,  lesson  test  grading  time  savings  occurred  daily  and  saved  small  amounts  of  time.  It  is 
unrealistic  to  think  that  the  time  savings  would  promote  productivity.  All  things  considered,  the  PLATO 
system  reduced  the  instructor’s  burden,  yet  the  time  savings  associated  with  this  reduction  had  no 
measurable  dollar  value  savings  in  this  environment. 

Potential  Savings.  Within  the  PLATOized  common  course  segment  environment,  instructors  had  few 
demanding  tasks.  Little  student-instructor  interaction  was  required,  and  the  instructors  felt  that  monitoring 
student  progress  would  just  disturb  the  students,  since  the  interaction  between  student  and  terminal  kept 
the  student  attentive.  In  a self-paced  environment,  or  a group-paced  design  with  a different  management 
structuring,  the  instructor  staff  could  be  reduced.  This  was  one  area  for  potential  savings.  Even  if  the 
number  of  individuals  required  in  the  PLATO  laboratory  to  assist  students  would  not  be  decreased,  it  was 
apparent  that  the  level  of  responsibility  could.  Many  of  the  exchanges  between  instructors  and  students  had 
no  training  function,  thus  instructor  assistants  could  have  easily  performed  adminstrative  and  management 
tasks. 

Additional  savings  would  result  from  increased  temiinal  utilization.  The  30  PLATO  terminals  were 
available  from  7.40  a.m.  to  10'. 00  p.m.  during  a period  designated  “prime  time.’’  Special  Purpose  Vehicle 
Repairman  course  students  interacted  with  the  terminals  for  approximately  25  to  30  percent  of  the 
available  prime  time.  Through  careful  management  of  the  resource  (sec  Management  section)  the  terminal 
utilization  could  be  at  least  doubled,  as  was  demonstrated  late  in  the  service  best  by  the  inclusion  of 
General  Purpose  Vehicle  Repairman  course  students  in  some  portions  of  the  training  program,  which  were 
in  common  with  the  Special  Purpose  Vehicle  Repaimian  courses. 

Another  area  where  PLATO  could  have  provided  savings  was  utilization  of  instructional  resources. 
Training  equipment  requirements  might  be  reduced  by  allowing  students  to  practice  tasks  on  the  terminal 
and  then  testing  on  real  equipment.  As  an  example,  students  were  required  to  wire  simple  circuits  on  a 
training  device.  Tire  activity  could  have  been  done  on  PLATO,  then  tested  with  real  devices;  the  more 
expensive  complex,  and  fragile  the  equipment,  the  greater  the  potential  savings.  To  illustrate,  the  student 
tuned  up  engines  with  an  engine  analyzer.  If  a simulation  were  made  which  allowed  the  student  to  attain  a 
desired  level  of  task  proficiency  before  actually  performing  the  real  task,  less  hands-on  time  would  be 
required  and  students  could  learn  the  task  more  quickly  with  less  chance  of  damaging  actual  equipment. 

A number  of  administrative  applications  could  be  performed  by  PLATO.  Student  records 
management  could  be  automatically  performed,  reducing  the  instructor's  responsibilities  and  possibly 
eliminating  a clerical  position.  Course  control  documents  might  be  developed  and  maintained  on  the 
PLATO  system.  This  would  expedite  their  preparation  and  revision.  If  PLATO  applications  were  expanded 
to  other  courses,  a terminal  situated  near  the  staff  agencies  charged  with  review  of  courseware,  records,  and 
control  documents  could  be  used  for  review  purposes. 

Other  Benefits.  It  is  difficult  to  place  a dollar  value  on  many  of  the  benefits  which  were,  or  could  be, 
associated  with  PLATO.  For  example,  how  much  money  was  saved  because  student  attitudes  were  higher 
towards  PLATO  than  programmed  texts?  Did  these  higher  attitudes  promote  efficiency?  It  was  not  possible 
to  determine  savings  from  these  benefits,  but  they  should  be  mentioned. 

In  many  cases,  PLATO  applications  improved  training  methods.  On-line  test  administration  and  test 
analysis  reduced  the  numberof  test -associated  tasks  (test  security  measures,  visual  inspection  of  test  forms 
for  pencil  marks,  annotating  individual  items  responses  for  each  student).  Instructors  were  pleased  about 
this  aspect  of  PLATO.  Updating  of  training  materials  was  much  easier,  changes  could  be  in  effect  the  same 
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day  that  the  need  became  apparent.  Hie  use  of  simulation  often  decreased  safety  risks  associated  with 
maintenance  tasks.  Students  who  practiced  reading  the  battery  hydrometer  and  interpreting  the  results  on 
PLATO  required  less  practice  in  the  laboratory.  Thus,  they  directed  more  of  their  attention  to  the  manual 
aspects  of  the  task  insuring  that  they  did  not  spill  battery  acid. 

Many  applications  may  be  derived  which  could  improve  training  practices.  With  respect  to  laboratory 
tasks  which  required  small  groups  of  students  to  perform,  it  was  doubtful  that  the  student  got  experience 
with  each  subtask.  Through  the  use  of  simulation  and  performance  testing  on  PLATO,  the  student  could 
attend  to  each  subtask.  Some  teaching  activities,  such  as  demonstrations,  could  be  improved:  the 
demonstration  could  not  always  be  performed  under  good  conditions.  A demonstration  performed  in  the 
Crash/Rescue  Vehicles  course  may  illustrate  the  point.  The  instructor  demonstrated  key  tasks  in  the 
operational  checkout  of  a fire  truck  system  to  a group  of  12  students.  Because  it  was  difficult  for  more 
than  four  or  five  students  to  view  the  procedure,  some  became  distracted  or  restless.  Consequently,  the 
value  of  the  demonstration  was  questionable.  Since  the  development  of  a training  aid  would  be  a long, 
expensive  process  and  a PLATO  lesson  with  interactive  displays  might  accomplish  the  intent  of  the  learning 
activities  in  far  less  time  with  greater  effect,  it  is  desirable  to  try  such  applications. 

The  flexibility  of  the  PLATO  system  could  allow  it  to  be  used  for  many  diversified  applications 
which  could  improve  instructional  methods.  It  was  noted  that  once  instructor  and  management  personnel 
accepted  and  became  involved  with  the  project  they  became  more  innovative;  i.e.,  thinking  of  ways  to 
improve  their  instructional  environment  through  PLATO  applications.  Again,  a dollar  value  cannot  be 
assigned  to  this  effect. 

Conclusion 

The  PLATO-based  Special  Purpose  Vehicle  Repairman  course  was  not  as  cost-effective  as  one  that 
would  use  programmed  texts  and  conventional  testing  procedures.  It  is  believed  though,  that  if  a serious 
effort  were  undertaken  to  optimize  system  usage  through  systematic  manipulation  of  real  and  potential 
benefits,  a PLATO-based  course  operation  would  cost  less  than  a non-CAl  environment  that  had  similar 
goals  and  desired  outcomes. 


V.  INSTRUCTIONAL  IMPACT 


Overview 

This  section  will  examine  the  impact  of  PLATO  on  vehicle  maintenance  training  conducted  in  the 
four  service  test  courses.  The  specific  aspects  to  be  reported  are  (a)  student  and  instructor  attitudes,  (b)  the 
impact  of  PLATO  on  daily  training  activities,  (c)  changes  in  Air  Force  regulations  necessary  to  incorporate 
PLATO,  and  (d)  the  effect  of  PLATO  on  potential  new  capabilities  within  vehicle  maintenance  training. 

Student  and  Instructor  Attitudes 

Introductum.  Measuring  student  attitudes  toward  training  was  necessary  to  obtain  an  indication  of 
how  well  the  instruction  was  matched  to  student  capabilities.  If  attitudes  are  positive,  it  is  more  likely  that 
a training  system  is  operating  with  acceptable  efficiency.  But,  it  student  attitudes  are  found  to  be  negative, 
it  is  possible  that  something  is  wrong. 

In  like  manner,  instructor  attitudes  can  be  an  indication  of  training  system  acceptability.  If  attitudes 
are  positive,  it  can  be  expected  that  instructors  will  work  with  and  improve  an  instructional  system.  On  the 
other  hand,  if  attitudes  are  negative,  it  is  highly  likely  that  the  system  will  fall  into  disuse  as  soon  as  the 
tryout  period,  regardless  of  its  outcome,  is  completed. 

For  this  portion  of  the  PLATO  evaluation,  a variety  of  attitudes  were  measured  in  order  to  cover  a 
broad  spectrum  of  student  and  instructor  reactions  to  introduction  of  the  new  system.  Surveys  were 
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developed  to  examine  attitude  changes  over  time,  to  detect  reactions  to  specific  features  of  the  training 
sytcni  (both  PLATO  and  non-PLATO),  and  to  compare  attitudes  of  students  at  Chanute  with  attitudes  of 
students  at  other  training  centers  who  had  not  experienced  PLATO. 

Student  Attitudes.  Student  attitudes  were  assessed  with  three  instruments.  Tire  first  questionnaire, 
designated  the  Short  Form  Survey,  was  designed  to  measure  changes  in  attitudes  toward  medium  of 
instruction  as  students  progressed  through  the  course.  A second  questionnaire,  called  the  Long  Form 
Survey,  was  developed  to  obtain  more  extensive  reactions  to  the  PLATO  medium  specifically.  The  final 
measure  of  student  attitudes,  the  Technical  Training  Survey,  was  an  instrument  developed  by  the  Personnel 
Research  Division  of  AFHRL.  This  questionnaire  was  employed  in  the  present  study,  primarily  to  compare 
the  attitudes  toward  technical  training  of  the  present  sample  with  those  of  the  entire  student  population  at 
Chanute,  as  well  as  with  students  at  other  ATC  training  centers. 

Short  Form  Survey.  Two  different  versions  of  the  Short  Form  Survey  were  constructed,  one  for 
students  who  experienced  PLATO  training,  and  one  for  students  in  the  non-PLATO  (NP)  condition.  These 
instruments  were  intended  to  be  parallel  forms  and  consisted  of  five  endorsement  items  and  two 
open-ended  items  (see  Appendix  B).  The  PLATO  version  and  the  NP  version  differed  basically  with  respect 
to  two  items  covering  medium  of  instruction.  PLATO  students  saw  items  referring  to  PLATO,  while 
non-PLATO  students  saw  items  referring  either  to  the  instructor  or  classroom  depending  upon  which  was 
more  appropriate. 

The  five  endorsement  items  were  scored  using  a Likert-type  format,  with  a scale  ranging  from  0 to  9. 
They  covered  five  aspects  of  the  training  situation-enjoyment  of  training,  perceived  effectiveness  of 
training,  satisfaction  with  principal  medium  of  instruction  (instructor  or  PLATO),  adequacy  of  the  content 
of  instruction,  and  perceptions  of  the  extent  to  which  training  was  a challenge.  Tire  open-ended  items 
requested  students  to  comment  on  what  they  liked  best  about  their  training  and  what  they  would  like  to 
change. 

The  Short  Form  Survey  measured  attitudes  at  a general  level  and  was  intended  to  permit  comparisons 
between  conditions  (PB,  CP,  and  NP)  at  different  points  in  time. 

Method.  During  the  early  stage  of  the  evaluation,  the  Short  form  was  given  to  all  students  four  times, 
once  in  each  of  the  four  common  blocks.  A total  of  41  students  in  the  PB  condition.  35  students  in  the  CP 
condition,  and  23  students  in  the  NP  condition  completed  all  four  administrations  of  the  survey  from  early 
January  1975  until  the  last  week  of  June  1975.  All  classes  beginning  after  May  1975  were  entered  into  the 
PB  condition.  At  this  time,  administration  in  every  block  ceased,  and  only  students  completing  the  fourth 
block  of  instruction  were  given  the  instrument.  Consequently,  data  were  collected  from  132  additional 
students  during  the  May  through  September  period  of  the  service  test  as  they  completed  the  PLATO-bascd 
portion  of  their  training. 

The  Short  form  was  administered  approximately  2 days  before  the  end  of  each  block  of  instruction. 
For  students  in  the  group-paced  PB  condition,  this  entailed  completing  it  at  7-  to  10-dav  intervals.  Students 
in  the  lock-step  CP  and  NP  conditions  received  the  questionnaire  after  regular  14-day  intervals.  All 
administrations  of  this  form  were  conducted  in  the  normal  classroom. 

Results.  For  the  purpose  of  comparing  CP  and  PB  condition  atti'  des  toward  PLATO,  the  five  items 
on  the  PLATO  questionnaire  were  summed  and  considered  to  be  a sca;c  When  including  the  NP  condition 
as  well,  only  items  2.  3.  and  4 seemed  to  logically  compare  media  across  the  three  conditions.  That  is,  the 
substitution  of  instructor  for  "PLATO,"  in  items  I and  5 was  viewed  as  keying  the  student  to  a particular 
medium,  whereas  the  connotations  of  “class”  (items  I and  5,  Appendix  B)  seemingly  related  to  the  entire 
instructional  milieu.  Therefore,  for  comparisons  involving  the  NP  condition  the  score  was  calculated  by 
summing  items  2,  3,  and  4.  The  internal  consistency  reliabilities  of  these  two  scales,  as  estimated  by 
Cronbach's  Alpha,  arc  reported  in  Table  21.  These  reliability  estimates  were  found  to  be  acceptable. 
Comparison  of  PB  and  CP  conditions  was  made  by  summing  all  five  responses  to  create  a scale  score  for 
each  student  on  each  of  the  four  administrations.  To  facilitate  analysis,  twenty  students  were  randomly 
selected  from  each  of  the  PB  and  CP  conditions  and  their  scores  were  analyzed  using  a 2 x 4 (condition  by 


Table  21.  Short  Form  Survey  Alpha 
Reliability  Estimates 


Reliability/ 

Number 

Block  of  Administration 

i 

2 

3 

4 

A.  Five-Item  Scale  (CP  and  PB  Conditions) 

a 

.817 

.856 

.889 

.900 

n 

104 

104 

194 

198 

B.  Three-Item  Scale  (NP,  CP,  ard  PB  Conditions) 

a 

.705 

.745 

.830 

.817 

n 

126 

126 

136 

219 

administration)  ANOVA  with  repeated  measures  on  the  last  factor.  Table  22  indicates  that  only  the 
administration  main  effect  was  significant.  As  Figure  3 shows,  the  mean  student  scale  scores  decreased  over 
the  four  administrations  of  the  survey.  Scale  scores  could  range  from  0 to  45;  thus,  a value  of  22.5  would 
be  considered  neutral.  Though  student  attitudes  decreased  across  administrations,  all  means  showed  a 
positive  orientation  toward  PLATO. 


Table  22.  Analysis  of  Variance:  Short  Form  Survey 
I PB  and  CP  Conditions) 


Source 

MS 

df 

F 

Between  Subjects 

Condition  (CP,  PB) 

726.76 

i 

2.79 

Sub  w/conditions 

260.02 

38 

Within  Subjects 

Administration  (14) 

406.60 

3 

10.86* 

Condition  x Administration 

16.91 

3 

.45 

Admin  x Sub  w/conditions 

37.45 

114 

*p  < .01. 


Similar  to  the  preceding  analysis,  scale  scores  for  a random  selection  of  twenty  students  in  all  three 
conditions  were  compared.  A 3 x 4 (condition  by  administration)  repeated  measures  ANOVA  was 
performed  on  scale  scores  for  the  three  items  which  addressed  the  principal  medium.  The  results  (Table  23) 
were  similar  to  the  preceding  analysis  in  that  only  the  administration  main  effect  attained  significance,  p < 
.05.  Again,  the  decreased  mean  scale  scores  across  administrations  (Figure  4)  appear  to  be  responsible.  Since 
the  neutral  point  for  this  scale  was  13.5,  all  attitude  means  were  again  in  the  positive  rang.'.  , 

Examination  of  the  open-ended  items  revealed  spontaneous  comments  about  the  positive  and 
negative  aspects  of  using  PLATO.  On  the  positive  side,  students  reported  liking  graphics  and  illustrations 
(particularly  those  involving  movement),  felt  the  programs  were  easy  to  understand,  and  were  pleased  with 
the  review  features  that  PLATO  employed.  They  also  frequently  reported  that  they  enjoyed  the  enrichment 
lessons  and  the  games  that  were  available  as  time  buffers  for  fast  students.  Many  positive  comments  were 
also  related  to  group-pacing. 

Negative  comments  mostly  concerned  the  PLATO  lessons.  Several  comments  indicated  that  wording 
of  questions  did  not  match  the  instruction  and  that  missed  questions  often  required  review— not  only  of 
material  related  to  the  question  but  of  additional  material  which  was  not  related.  Many  comments  were 
directed  at  the  exactness  of  the  answers  required.  Close  answers  were  often  judged  wrong,  which  was 
frustrating  and  annoying.  Students  reported  difficulty  reading  microfiche  images,  particularly  when 
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'■'ifUirc  3.  Comparison  of  short  Form  Survey  results  for  the  PLATO-Using 

evaluation  conditions. 


Tabic  23.  Analysis  of  Variance:  Short  Form  Survey 
( PR.  CP.  and  .VP  Conditions) 


Sou  ret 

MS 

df 

F 

Between  Subjects 

Condition  (PB,  NP,  CP) 

134.47 

2 

1.77 

Sub  w/groups 

75.82 

57 

Within  Subjects 

Administration 

142.62 

3 

3.24* 

Condition  x Administration 

74.81 

6 

1.70 

Admin  x Sub  w/groups 

44.08 

171 

*p  < .05. 
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Block  of  Administration 

l-'igure  4 Comparison  of  student  attitudes  towards  principal  medium 

of  instruction. 


computer-generated  words  or  graphics  were  also  displayed  on  tire  plasma  panel  at  the  same  time.  The  only 
negative  statements,  not  relative  to  PLATO,  concerned  the  hardness  of  the  chairs. 

Discussion.  Analysis  of  the  Short  Form  Surveys  showed  several  marked  consistencies  among  student 
responses  toward  PLATO  as  a training  medium.  First,  attitudes  toward  PLATO  remained  positive 
throughout  the  6-  to  8-wcck  exposure  to  the  system.  Students  enjoyed  using  PLATO  even  after  20  or  more 
hours  of  contact  with  it.  As  might  be  expected,  the  novelty  diminished  over  time  which  may  have 
accounted  in  part  for  the  positive  attitude  reduction  over  the  period.  Another  contributing  factor  may  have 
been  the  state  of  the  lessons  during  the  data  collection  period.  Most  lessons  had  not  been  validated; 
consequently  they  contained  technical  errors  and  were  subject  to  presentation  difficulties.  Increasing 
exposure  to  the  unvalidated  lessons  may  have  lowered  attitudes  toward  PLATO. 

As  Figure  3 indicates,  student  attitudes  seemed  more  consistent  toward  PLATO  across 
administrations  than  they  were  toward  instructors.  Just  prior  to  the  evaluation  period,  several  new 
instructors  joined  the  Vehicle  Maintenance  Branch.  Since  their  teaching  skills  were  not  fully  developed,  it  is 
logical  to  assume  that  student  contact  with  them  could  have  caused  student  attitudes  to  become  less 
positive.  Even  though  not  wholly  appropriate  (no  significant  main  effect  was  noted)  two  ixtst  hoc  mean 
contrasts  were  performed.  Using  the  Tukey  procedure  (Kirk,  1968,  p.  268)  the  combined  CP  and  PB  mean 
was  found  to  be  significantly  greater  (p  < .05)  than  the  NP  mean  for  each  of  the  first  two  administrations. 
The  new  instructors  taught  the  first  two  blocks  of  instruction,  where  these  possible  attitude  differences 
between  NP  and  combined  PLATO  students  were  found.  Also,  a popular  instructor  taught  each  of  the  NP 
condition  classes  in  block  3 and  a less  popular  instructor  taught  block  4.  The  Tukey  procedure  was  again 
used  to  contrast  the  four  NP  condition  means.  A significant  attitude  difference  was  found  between  blocks  2 
and  3,  and  the  decline  from  block  3 to  block  4 was  also  significant.  Course  critique  information  was 
consistent  with  these  quantitative  results. 
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Not  surprisingly,  all  data  indicated  that  in  a training  environment  where  the  instructor  is  a principal 
medium,  student  attitudes  will  be  substantially  influenced  by  the  characteristics  of  the  instructor. 

Content  analysis  of  the  open-ended  comments  indicated  that  while  students  liked  many  aspects  of  the 
PLATO  system,  there  was  still  room  for  improvement  in  the  Chanute  lessons.  Again,  it  should  be 
remembered  that  these  lessons  were  in  the  formative  evaluation  stage,  and  most  were  validated  after 
administration  of  the  surveys  used  to  assess  condition  differences.  It  was  determined  that  in  each  of  the 
conditions  students  liked  their  principal  medium  of  instruction.  Also,  standardized  presentation  media, 
such  as  PLATO,  provided  a consistency  to  student  attitudes.  Finally,  student  orientation  toward  a medium 
remained  positive,  but  declined  as  the  students  became  more  familiar  with  the  medium. 

Long  Form  Survey.  Attitudes  of  students  toward  specific  features  of  PLATO  were  assessed  by  means 
of  a specially  constructed  66-item  inventory  designated  the  Long  Fomi.  This  instrument  was  developed  to 
obtain  more  in-depth  reactions  than  was  possible  with  the  Short  Form  Survey.  Forty-five  of  the  Long  Form 
items  were  adapted  from  a scale  developed  by  Brown  (1966)  and  were  used  to  measure  attitudes 
specifically  related  to  working  with  the  PLATO  medium.  Of  the  remaining  items,  six  were  directed  at 
attitudes  toward  various  media  (movies,  PLATO,  programmed  texts);  twelve  employed  a modified  semantic 
differential  (c.g.,  PLATO  is  boring,  exciting,  fun)  and  two  were  related  to  perceptions  of  system  reliability. 
A final  item  measured  overall  satisfaction  with  PLATO.  All  but  the  last  three  items  were  measured  by  a 
10-point  agree  disagree  scale  (0  to  9).  The  last  three  items  measured  how  much  time  a person  had  difficulty 
with  PLATO  or  liked  it.  They  used  a percentage  scale  of  0 to  9 representing  10  to  100  percent  of  the  time 
(see  Appendix  C). 

Method.  The  Long  Form  was  administered  once,  at  the  end  of  the  fourth  block  of  instruction  to 
students  in  the  PB  and  CP  conditions.  From  January  1975  through  September  1975,  43  students  in  the  CP 
condition  and  157  students  in  the  PB  condition  completed  this  survey. 

Responses  of  all  students  were  combined  and  the  questionnaire  was  factor  analyzed  to  determine  the 
attitude  areas  sampled.  This  analysis  revealed  five  general  factors.  These  factors,  the  items  in  each,  and 
their  loadings  are  presented  in  Appendix  D. 

The  first  factor  consisted  of  those  items  related  to  PLATO  and  was  labeled  PLATO  Training.  The 
second  factor  was  difficult  to  classify;  however,  the  items  were  generally  related  to  the  annoying, 
frustrating  aspects  of  working  with  PLATO,  designated  Training  Frustration.  Items  comprising  the  third 
factor  related  to  stressful  aspects  of  the  training  environment  and  were  therefore  labeled  Training  Stress. 
The  fourth  factor  contained  references  to  media  other  than  PLATO  which  were  employed  in  daily  training 
and  was  titled  “Non-PLATO  Media.”  The  final  factor  was  related  to  the  humanistic  aspects,  or  lack  thereof, 
in  a computer-based  training  setting.  Accordingly,  this  fifth  factor  was  labeled  “Mechanistic  Training.” 

Results.  Based  on  tire  factor  analysis,  each  of  the  five  dimensions  was  considered  to  be  a j.nle.  and 
scale  scores  were  calculated  for  each  person.  A person’s  item  responses  were  summed  yielding  a raw  total 
for  each  (actor.  In  order  to  keep  the  factor  scores  comparable  across  scales,  the  sums  of  the  item  responses 
were  divided  by  the  number  of  items  appearing  on  a particular  scale.  The  means  of  these  scale  scores  ate 
presented  in  Table  24.  On  the  whole,  it  can  be  seen  that  students  tended  to  be  positive  toward  PLATO 
instruction,  did  not  feel  frustrated  or  overly  stressed,  and  did  not  find  the  training  to  be  mechanistic. 
Comparison  of  the  PB  and  CP  conditions  by  /-test  indicated  that  no  significant  differences  were  present  for 
any  of  the  factors. 

Since  the  system  was  constantly  improving  over  the  course  of  the  survey,  it  was  hypothesized  that 
later  PLATO-Based  (LPB)  students  would  have  more  positive  attitudes  than  earlier  PLATO-Bascd  (EPB) 
students.  Additional  analyses  tailed  to  uphold  this  hypothesis.  In  fact,  the  opposite  was  observed  on  Factor 
I (Attitude  toward  PLATO  instruction),  the  means  being 6. 31,  and  5.60,  for  the  EPB  and  LPB  conditions, 
respectively.  Using  a t-test  (t  -2.14,  p - 03,  df=  143)  this  difference  was  significant.  This  decline  may 
have  been  the  result  of  an  initial  Hawthorne  effect,  or  a reduction  in  instructor  attitudes,  which  will  be 
discussed  later. 
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Table  24.  Mean  Responses  to  Attitude 
Factors  by  Students 
f PH  and  CP  Conditions, 

Scale  = 0 to  9) 


Factor 

PB 

(N  = 153) 

CP 

(N  = 42) 

PLATO  Instruction 

5.79 

5.27 

Frustration 

4.74 

3.58 

Training  Stress 

3.44 

3.56 

Non-PLATO  Media 

5.31 

5.33 

Mechanistic  Training 

2.75 

3.17 

Examining  specific  items  relating  to  media  preference  revealed  that,  in  the  PB  condition,  PLATO 
lessons  were  the  most  preferred  medium  next  to  laboratory  instruction.  CP  students,  on  the  other  hand,  did 
not  hold  PLATO  lessons  in  such  high  regard,  rating  them  least  preferred  of  all  media.  The  difference 
between  means  on  this  item  was  statistically  significant  (t(196)  = 2.28,  p = .02).  A graphic  representation  of 
the  mean  ratings  of  the  various  media  is  presented  in  Figure  5.  The  only  other  notable  difference  between 
i conditions  was  the  higher  preference  of  the  CP  students  for  laboratory  instruction  (t(l  96)  = 2.45,  p = .02). 

Relative  differences  between  conditions  in  terms  of  the  semantic  differential  items  are  graphically 
displayed  in  Figure  6.  In  general,  students  in  the  PB  condition  had  a tendency  to  rate  these  items  about  the 
same  as  CP  students.  Treating  the  twelve  responses  as  a scale,  no  significant  differences  between  conditions 
were  found.  Looking  at  specific  items,  one  major  difference  was  that  PB  students  perceived  PLATO  to  be 
more  frustrating  (t(  196)  = 2.82,  p<  .01).  This  difference  may  be  related  to  the  fact  that  PB  students  had 
more  interaction  with  PLATO.  The  only  other  significant  difference  was  that  PB  students  were  more  likely 
to  report  that  PLATO  made  them  feel  proud  of  their  accomplishments  (t(l 95)  = 2.38,  p<  .02).  The 
group-pacing  mode  experienced  by  PB  students  may  have  accounted  for  this  difference,  since  group-pacing 
gave  students  more  immediate  feedback  on  the  results  of  their  efforts  than  did  lock-step  course  design. 

To  determine  the  relationship  between  system  reliability  and  attitudes  toward  PLATO  lessons,  two 
items  dealing  with  system  reliability  were  correlated  with  the  item,  “As  a student  I do  my  best  work  with 
PLATO  lessons”  (item  2,  section  III).  The  first  reliability  item  was  positively  worded  and  asked  how  often 
the  system  worked  when  attempting  to  use  it;  the  second  reliability  item  was  negatively  worded,  asking 
how  often  system  interruptions  made  the  respondent  want  to  stop  using  PLATO  (items  19  and  20,  section 
III).  If  attitude  and  system  reliability  were  related,  one  should  find  a positive  correlation  between  attitudes 
toward  PLATO  lessons  and  the  first  reliability  item.  Conversely,  a negative  correlation  would  be  expected 
with  the  second  reliability  item. 

As  expected,  these  relationships  were  found  to  exist.  Correlation  between  perceived  reliability  and 
attitude  toward  PLATO  lessons  was  .40,  significantly  different  from  zero  at  the  .01  level.  Correlation 
between  the  negatively  stated  reliability  item  and  attitude  was  -.42,  again  statistically  significant.  Thus,  the 
more  frequently  students  had  trouble  with  PLATO  hardware,  the  more  negative  their  attitudes. 

Discussion.  The  in-depth  investigation  of  attitudes  toward  PLATO  showed  that  the  system  was 
perceived  as  an  acceptable  part  of  the  training  environment.  Combining  PLATO  with  the  group-paced 
training  mode,  though  tending  to  increase  frustration  levels,  did  not  reduce  positive  student  attitudes 
toward  their  training.  Moreover,  the  fact  that  CP  students  rated  PLATO  lessons  as  the  least  preferred  of  all 
media  is  understandable  in  that  the  lessons  were  not  validated  and  the  CP  instructors  openly  expressed  their 
concern  over  the  effectiveness  of  the  PLATO  lessons  (see  discussion  of  course  effectiveness).  Finally, 
hardware  reliability  appeared  to  exert  a maior  influence  on  how  students  perceived  the  effectiveness  of  the 
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Figure  6.  Graphic  representation  of  the  mean  semantic  differential  ratings  of  PLATO  (PB  and  CP  conditions). 
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system,  and  as  a result  should  be  given  close  attention  in  all  decisions  concerning  future  use  of  CAI  in 
general,  or  PLATO  specifically. 

Technical  Training  Survey.  This  instrument  was  developed  by  the  Personnel  Research  Division  of 
AFHRL  for  tire  purpose  of  uncovering  altitudinal  or  motivational  factors  connected  with  attrition  from 
technical  training.  It  was  administered  to  12.666  students  across  An  Training  Command  in  1974  and  1975. 
Twelve  dimensions  were  measured  using  121  items.  Factor  analysis  confirmed  the  construct  validity  of  the 
scales  (Vitola,  personal  communication).  A brief  description  of  the  instrument  follows;  the  entire  survey  is 
included  as  Appendix  E. 

The  first  three  scales  were  motivational  in  nature,  consisting  of  Instrumentality  (the  extent  to  which 
the  respondent  saw  good  performance  in  technical  training  to  be  instrumental  in  obtaining  valued 
outcomes);  Valence  (the  attractiveness  of  potential  outcomes  associated  with  good  performance  in 
technical  training),  and  Valence  of  Performance  (the  multiplicative  combination  of  instrumentality  and 
Valence  summed  over  all  outcomes).  These  concepts  come  from  the  Expectancy -Valence  formulations  of 
Vroom  ( 1 964)  and  I-awler  (1971 ).  Valence  of  Perfonnance  can  be  considered  a rough  measure  of  individual 
motivation,  since  motivation  is  theoretically  a partial  function  of  the  strength  of  the  reward  structure  in  a 
situation  and  the  extent  to  which  people  perceive  rewards  as  a consequence  of  good  performance. 

Tire  next  seven  scales  covered  opinions  about  particular  aspects  of  the  training  situation:  Instructor 
Technical  Competence  (the  extent  to  which  the  instructor  was  viewed  as  knowing  how  to  present 
instruction);  Instructor  Personal  Relations  (the  extent  to  which  the  instructor  was  seen  as  supportive  and 
considerate):  Fellow  Students  (the  extent  to  which  contacts  with  fellow  students  promoted  a harmonious 
learning  environment);  Organization  Control  (perceived  degree  of  control  exerted  by  the  organization); 
Training  Stress  (amount  of  pressure  felt  by  the  student);  Training  Materials  (perceived  adequacy  of 
materials,  methods,  and  their  utilization);  and  Physical  Setting  (satisfaction  with  environmental 
factors-vcntilation,  heat,  light,  work  space).  Finally,  two  short  scales  covered  Global  Training  Satisfaction 
and  Career  Choice  Satisfaction.  These  measures  provided  another  set  of  altitudinal  data  from  which  to  draw 
inferences  about  the  impact  of  PLATO  training. 

Method.  Two  administrations  of  the  survey  were  made  as  part  of  the  service  test.  These  occurred 
during  block  1 and  block  4 for  students  in  all  conditions.  Comparisons  between  conditions  across 
administrations  were  possible,  as  well  as  comparisons  between  PLATO  service  test  students  and  students 
across  the  entire  Air  Training  Command. 

Results.  Eight  of  the  12  scale  scores  were  analyzed  by  a 4 x 2 repeated  measures  ANOV A,  condition 
being  the  first  factor  and  administration  the  second.  Significant  differences  in  attitudes  can  be  summarized 
as  follows. 

For  all  the  motivational  variables,  there  were  significant  administration  effects,  but  no  differences 
between  conditions;  i.c.,  means  for  all  conditions  tended  to  decline  across  administrations.  This  was 
especially  noticeable  for  Instrumentality.  A similar,  but  less  pronounced  decline,  occurred  for  Valence. 
Since  Valence  of  Performance  was  multiplicative  combination  of  the  preceding  variables,  it  also  dropped. 

Both  scales  dealing  with  the  instructor-instructor  Technical  Competence  and  Instructor  Personal 
Relations -showed  significant  interaction  effects.  Results  of  these  two  ANOV  As  are  presented  in  Tables  25 
and  26;  associated  figures  which  depict  the  relationships  graphically  are  Figures  7 and  8.  In  answering  the 
questions  dealing  with  the  instructor,  students  in  the  PLATO  condition  were  told  to  respond  as  if  the 
PLATO  system  were  the  instructor.  Hence,  these  results  permit  a comparison  between  attitude  toward  real 
instructors  and  toward  the  PLATO  system.  Post-hoc  analysis  by  Scheffe  method  showed  that  the  means  for 
the  two  PLATO-using  conditions,  taken  together,  were  significantly  lower  than  for  the  non-PLATO 
conditions  on  Administration  2. 

Attitudes  toward  the  physical  setting  also  declined  between  the  two  administrations,  but  there  were 
no  changes  across  time  or  differences  between  conditions  on  either  Global  Training  Satisfaction  ot  Career 
Choice  Satisfaction.  Due  to  an  error  in  coding  the  raw  data,  repeated  measures  analyses  could  not  be 
performed  on  four  scales- Fellow  Students,  Organizational  Control,  Training  Stress,  and  Training  Materials. 
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Table  25.  Analysis  of  Variance: 
Instructor  Technical  Competence 


Source 

MS 

df 

F 

Between  Subjects 

Condition  (PB,  CP,  NP,  BL) 

238.14 

3 

2.56 

Subj  w/conditions 

93.18 

76 

Within  Subjects 

Administration  (1,2) 

819.02 

1 

22.52* 

Condition  x Administration 

330.08 

3 

9.08* 

Admin  x Sub  s/conditions 

36.36 

76 

*p  < .01. 

Table  26.  Analysis  of  Variance: 
Instructor  Personal  Relations 

Sourca 

MS 

df 

F 

Between  Subjects 

Condition  (PB,  CP,  NP,  BL) 

276.57 

3 

3.38* 

Subj  w/conditions 

81.70 

76 

Within  Subjects 
Administration  (1,2) 

1248.81 

1 

37.95* 

Condition  x Administration 

160.66 

3 

4.88* 

Admin  x subj  w/conditions 

32.90 

76 

*p  < .01. 
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Figure  7.  Mean  values  of  instructor  technical  competence  by  condition  and  administration. 
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Figure  8.  Mean  values  of  instructor  personal  relations  by  condition  and  administration. 


The  problem  of  comparing  attitudes  of  students  in  the  present  study  with  those  of  all  students  at 
Chanute  and  all  technical  training  students  was  approached  in  a straightforward  way  through  the  use  of 
confidence  intervals.  In  the  present  case,  there  was  a set  of  means  from  relatively  small  samples  (n  ranged 
from  20  to  158  in  the  four  conditions)  and  two  other  sets  of  means  coming  from  very  large  samples  (n  = 
2,484  and  n = 12,666).  These  latter  sets  may  be  taken  as  parameter  values  since  they  ostensibly  included 
the  populations  of  interest.  Given  the  observed  means  for  the  four  conditions  of  the  PLATO  service  test, 
and  an  estimate  of  the  population  standard  deviation,  confidence  intervals  around  the  population 
parameters  were  constructed  in  order  to  delimit  the  boundaries  within  which  samples  like  these  could  have 
been  drawn.  Then,  the  question  became:  Was  the  population  parameter  bracketed  by  the  sample  confidence 
interval?  If  parameter  values  failed  to  fall  within  these  boundaries,  an  observed  mean  was  accepted  as 
coming  from  a different  population  and  an  actual  difference  was  present,  given  a .05  probability  of  error. 

Means  and  confidence  intervals  for  the  two  administrations  are  given  in  tabular  form  in  Appendices  F 
and  G.  These  data  revealed  some  scattered  differences  from  population  parameters.  One  consistent 
tendency  in  the  first  administration  was  for  PB  students  to  have  generally  higher  motivation,  attitudes,  and 
satisfaction.  In  the  second  administration  of  the  survey,  however,  these  tendencies  reversed  for  the  PB 
group  with  the  exception  of  satisfaction  and  Instrumentality  which  remained  higher  than  the  Chanute  or 
ATC  mean  ratings.  In  addition,  the  second  administration  found  CP  students  considerably  lower  than 
Chanute  and  ATC  norms  on  nearly  all  the  dimensions. 

Discussion.  Observed  differences  on  the  dimensions  covered  by  the  Technical  Training  Survey  tended 
to  support,  in  general,  the  attitude  finding  reported  earlier.  Again,  there  was  a persistent  tendency  for 
attitudes  to  decline,  yet  to  remain  slightly  positive. 

The  most  plausible  interpretation  for  the  Instrumentality  decline  was  that  students  tended  to  perceive 
reality  more  correctly  after  experience  in  the  technical  training  environment.  There  were,  in  fact,  very  few 
real  connections  between  valued  outcomes  and  performance  in  technical  training.  The  naive  student 
probably  failed  to  appreciate  this  fact  until  he  had  had  more  exposure  to  the  realities  of  Air  Force  training. 
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The  decline  in  valence  of  outcomes  was  more  difficult  to  explain,  but  previous  research  in  the  motivation 
area  has  noted  a tendency  for  Valence  to  be  correlated  empirically  with  Instrumentality  despite  their 
theoretical  independence. 

The  decline  in  attitudes  toward  the  instructor  noted  only  for  the  PLATO  conditions  was  consistent 
with  the  previously  reported  drop  in  attitudes  toward  the  PLATO  system  over  time. 

Instructor  Attitudes.  Opinions  of  instructors  using  PLATO  were  assessed  by  means  of  a 40-item 
Likert-type  questionnaire  designed  to  examine  acceptance  or  rejection  of  the  PLATO  system  and  its  impact 
on  perceptions  of  their  job.  Of  the  40  items,  38  were  agree/disagrec  statements  referring  to  characteristics 
of  PLATO,  ISD,  and  job  satisfaction  (sec  Appendix  H).  Responses  were  scored  on  a 0 to  9 scale.  The  two 
remaining  items  covered  time  spent  using  the  PLATO  system. 

Method.  The  questionnaire  was  administered  in  February  1975  and  again  in  July  1975.  Eighteen 
instructors  participated  in  the  first  administration  and  thirteen  in  the  second,  with  ten  instructors 
completing  the  survey  both  times.  Five  a priori  dimensions  were  constructed  and  items  written  to  tap  these 
dimensions:  (a)  attitude  toward  the  PLATO  system  as  a whole,  (b)  quality  of  the  PLATO  lessons  as  a 
medium  of  instruction,  (c)  attitude  toward  the  job  of  instructor,  (d)  attitude  toward  ISD,  and  (e)  PLATO 
affect  as  measured  by  the  modified  semantic  differential  scale.  In  addition  to  these  dimensions,  instructors 
made  ratings  of  various  instructional  media  using  the  same  items  which  were  present  on  the  student  survey. 

Results.  Table  27  presents  the  items  included  on  the  five  derived  scales  plus  an  estimate  of  scale 
reliability  (Cronbach’s  alpha)  where  possible.  The  longer  scales  demonstrated  acceptable  reliability. 


Table  27.  Reliability  of  the  Instructor 
Survey  Subscales 


Dimension 

Item  Numbers 

Alpha 

PLATO  System 

3,  16,  17,  18.20, 
22,35,36,39,40 

.88 

PLATO  Lessons 

6,  19,37,  38 

.80 

Job  Satisfaction 

1.2 

ISD 

4 

. . 

PLATO  Affect 

23  to  34 

.93 

Table  28  presents  means  and  t-tests  for  the  attitudinal  data  analyzed  as  repeated  measures  with 
correlated  observations.  Item  responses  have  been  averaged  so  that  4.5  represen  a neutral  response,  except 
for  the  PLATO  affect  scale  where  54  was  the  neutral  point.  A drop  in  attitude  toward  the  PLATO  system 
and  PLATO  lessons  was  observed  only  for  the  PLATO  affect.  Inspection  of  the  means,  however,  shows  that 
even  after  the  highly  favorable  initial  attitude  had  worn  off,  the  general  orientation  of  instructors  was 
slightly  better  than  neutral. 

When  the  pattern  of  positive  and  negative  responses  to  the  semantic  differential  items  was  considered, 
a clearly  significant  difference  was  detected  (Figure  9 and  Table  29).  A Chi-Square  test  was  performed  on 
the  individual  responses  between  administrations,  X*0)=  38.22,  p<  .001.  This  result  confirmed  the  fact 
that  a major  shift  in  feelings  about  the  PLATO  medium  had  occurred  over  the  5-month  interval.  To  make 
the  situation  more  evident.  Table  30  was  constructed  which  lists  the  means  of  the  semantic  differential 
items  for  both  administrations  and  the  net  change  between  administrations.  A sign  test  confirmed  that  this 
result  was  not  likely  to  have  occurred  by  chance,  x20  ) = 1 2.0,  p < .01 . 

Instructor  preferences  for  various  teaching  media  are  shown  graphically  in  Figure  10.  Little  change 
can  be  noted  between  administrations.  The  only  differences  which  approached  customary  statistical 
significance  (p  < .10)  were  for  PLATO  lessons  which  dropped  approximately  .5  scale  point,  and  study 
guides  and  workbooks  which  gained  about  2 points. 
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Table  29.  Positive  and  Negative  Responses  on  Semantic 
Differential  Items 


> < .001. 


Table  30.  Means  and  Net  Change  for  Items  23-34 


All  12  items  changed  in  negative  direction 

Sign  Test  Results 
Responses 


Admin  2 


12 

0 

4 

8 

XJ  = 12.0  p<  .001. 
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Figure  10.  Graphic  representation  of  instructor  ratings  of  effectiveness  of  vs  ous 
media  on  two  administrations  of  the  instructor  attitude  survey. 
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Disucussion.  To  summarize,  instructor  attitudes  were,  in  the  beginning,  positive  toward  PLATO  as  an 
instructional  medium  which  may  be  attributable  to  initial  enthusiasm.  Their  reappraisal  after  5 months 
appeared  to  indicate  a dissatisfaction  with  their  less  active  role  in  the  instructional  environment.  The 
instructor  perceived  PLATO  as  somewhat  depersonalizing,  a catalyst  for  decreased  role  activity,  and  as  a 
suppressing  agent  for  a sense  of  accomplishment  and  self-fulfillment. 

These  findings  point  to  a need  to  increase  instructor  involvement  in  PLATO-based  instruction.  It  is 
conceivable  that  reducing  the  total  number  of  instructors,  associated  with  operating  PLATO,  may  increase 
their  activity  level  and  improve  morale.  Perhaps  an  arrangement,  whereby  a few  instructors  are  assigned 
specifically  to  PLATO  training  (wherein  they  become  experts  on  the  lessons,  routers,  data  collection 
procedures)  and  the  remaining  instructors  are  assigned  specifically  to  laboratory  training  may  be  a possible 
solution  to  this  problem.  In  any  case,  the  complexity  of  operating  PLATO  at.  or  near,  its  potential  wQl 
always  require  a nucleus  of  highly  trained  instructors,  further  discussion  of  instructor  utilization  will  be 
presented  in  evaluation  of  management. 

The  fact  that  a drop  in  instructor  attitude  was  detected  in  this  study  should  not  obscure  the  fact  that 
instructor  attitudes  were  at  least  neutral  in  the  most  recent  administration  of  the  survey.  Acceptance  of  the 
system  was,  in  the  final  analysis,  satisfactory,  though  perhaps  not  as  strongly  positive  as  had  been 
anticipated. 

Impact  on  Training  Activities 

Procedure.  To  determine  the  impact  of  PLATO  on  daily  training,  observations  of  classroom  and 
laboratory  activities  in  the  Special  Purpose  Vehicle  Repairman  courses  were  made  on  a random  basis.  Two 
observers  visited  classrooms  individually  at  irregular  times.  They  were  given  check  sheets  containing  alert 
items.  In  general,  their  guidance  was  to  observe  the  instruction  and  note  anything  out  of  the  ordinary.  Most 
frequently,  the  observers  made  extensive  notes  on  separate  sheets  of  paper.  The  two  principal  observers 
were  instructors  on  loan  to  the  PLATO  project  for  90  days.  They  had  extensive  experience  as  instructors 
and  could  recognize  problems  where  non-instruciors  might  not.  Over  a period  of  3 months,  they  spent 
approximately  40  hours  in  classroom  observation. 

These  measures  involved  a certain  amount  of  risk  that  observer  presence  would  affect  student 
behaviors,  but  after  several  sessions,  the  students  appeared  to  relax  and  pay  little  attention  to  the  observer. 
There  still  was  no  assurance  that  the  act  of  observing  the  students  did  not  influence  the  instruction,  but 
since  the  objectives  of  the  observations  were  not  clear  to  the  students  or  the  instructors,  reactivity  was  not 
considered  to  be  of  great  consequence. 

In  general,  the  principal  object  of  the  observations  was  to  obtain  an  insight  into  the  way  instruction 
was  presented  in  a conventional  classroom  setting  and  how  it  might  compaie  with  instruction  presented 
through  the  PLATO  system.  The  observations  were  organized  according  to  specific  areas  so  that,  when 
appropriate,  comparisons  could  be  made  between  conventional  instruction  and  PLATO  instruction.  The 
areas  were:  (a)  conduct  of  training,  (b)  student  interaction,  (c)  instructional  content,  and  (d)  instructional 
styles. 

Results  and  Discussion 

Conduct  of  Training.  Since  PLATO,  as  used  at  Chanute,  was  a resource  limited  to  29  terminals  for 
training,  a finite  amount  of  ECS,  and  operational  availability  from  0740  to  2200  hours,  it  was  necessary  to 
introduce  new  operating  procedures  to  accommodate  it.  The  major  changes  involved  operating  the  PLATO 
portion  of  all  four  courses  on  B shift  (1200  to  1800  hours).  A schedule  was  posted  at  the  entrance  of  the 
PLATO  laboratory  and  each  instructor  could  reserve  blocks  of  time  and  terminals  for  his  class  to  work  on 
PLATO  lesson  material.  Though  unprecedented,  the  system  appeared  to  have  worked.  Instructors,  however, 
complained  of  problems  in  getting  access  to  terminals  and  found  that  this  procedure  complicated  their  job. 

During  peak  loads  on  the  system,  there  would  occasionally  be  insufficient  ECS  for  all  students  in  the 
PLATO  laboratory  to  be  in  different  lessons  at  the  same  time.  This  required  PLATO  instruction  to  be 
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interrupted  for  some  classes  and  replaced  by  other  instruction  until  ECS  was  available.  Such  interruptions 
and  substitutions  were  unique  to  use  of  the  PLATO  medium. 

Problems  with  computer  operating  times  and  ECS  limitations  occurred  because  the  PLATO  system 
was  not  an  Air  Force  resource  and,  as  such,  there  was  not  direct  control  over  when  it  operated  or  *he 
priority  for  ECS.  This  feature  was  perhaps  the  single  most  serious  drawback  concerning  utilization  of 
PLATO  in  the  current  study. 

Student  Interaction.  Student  interaction  was  perhaps  the  richest  area  provided  by  the  observations. 
For  the  most  part,  conventional  classroom  activities  centered  around  instructor  presentations  (lectures). 
Student  interaction  with  these  lectures  varied  depending  upon  the  kinds  of  students  and  the  complexity  of 
the  subject  matter  being  presented.  At  times,  when  student  discussion  was  absent,  many  students  were 
observed  to  become  rather  quiescent,  almost  to  the  point  of  falling  asleep.  Any  interruption  in  the  routine 
lecture,  however,  became  a welcome  diversion.  Whenever  instruction  required  the  students  to  stand  or  move 
about,  they  seemed  much  more  alert. 

Instructor  questioning  of  the  students  took  two  forms.  The  instructor  either  asked  general  questions 
and  waited  for  a volunteered  answer,  or  he  asked  a specific  student  for  an  answer.  In  the  first  case,  if  the 
question  was  difficult  or  the  instruction  was  unclear,  it  would  often  be  followed  by  an  embarrassed  silence. 
If  easy,  several  students  would  attempt  to  answer  at  the  same  time.  In  either  case,  the  instructor  could  only 
sample  the  understanding  of  the  class  by  questioning  one  student.  Further,  by  repeated  questioning  on  a 
random  basis,  the  instructor  fell  victim  to  the  chance  of  overlooking  some  students.  This  was  observed  on 
one  occasion  where,  over  a 4-hour  period,  three  students  out  of  eleven  either  asked  questions  or  were 
questioned  10,  12,  and  15  times,  respectively.  Interactions  for  the  remaining  students  averaged  around  four 
to  five.  The  general  conclusion  was  that  programmed  instruction  of  PLATO  can  reduce 
non-representativeness  of  questioning  and  space  student  interaction  more  equitably. 

On  the  other  hand,  there  is  something  positive  to  be  said  for  lecture/discussion-type  instruction.  It 
was  observed  that  a single  question  from  a student  during  a lecture  could  provoke  further  questions  from 
other  students.  These  interactions  then  allowed  the  instructor  to  adapt  his  presentation  to  the  needs  of  the 
class.  In  programmed  instruction,  even  CAI,  this  generalized  adaptation  is  difficult  to  achieve.  If  a student 
has  a question  during  a CAI  lesson,  he  may  call  the  instructor  for  an  expalanation,  but  while  this  may 
answer  his  question,  none  of  the  other  students  can  benefit  from  hearing  the  interaction  as  they  would  in 
the  classroom.  Ideally,  CAI  lessons  would  be  revised  as  such  problems  arose.  In  practice,  however,  this  ideal 
is  hard  to  realize.  First,  each  incidence  of  such  a question  must  be  recorded.  If  the  question  frequently 
arises,  it  would  merit  changing  the  CAI  lesson.  Otherwise,  such  a change  would  not  be  warranted. 
Attempting  to  keep  a record  of  questions,  while  in  the  midst  of  monitoring  the  progress  of  ten  or  eleven 
students  through  a CAI  lesson  would  be  a difficult,  if  not  impossible,  task  for  a single  instructor.  Hence,  the 
most  that  can  be  done  without  expending  a major  effort  is  to  refine  a lesson  to  teach  to  a specified  teaching 
objective.  It  would  seem,  therefore,  that  on  a larger  scale  than  simple  training  objectives,  stand-up 
instruction  can  be  more  adaptive  than  the  current  use  of  CAI  at  Chanutc. 

Instructional  Content.  Observing  the  instruction  revealed  that  students  were  strongly  goal  oriented 
and  attended  most  closely  to  instruction  directly  associated  with  that  goal.  For  example,  when  instruction 
dealt  with  a maintenance  task  wherein  actual  equipment  was  used  in  a nearly  natural  setting  (the 
laboratory),  the  students  appeared  most  interested  and  involved.  On  numerous  occasions,  when  students 
actually  engaged  in  hands-on  work,  there  was  total  involvement.  When  the  classroom  instructor  told  of  an 
experience  he  had  “in  the  field”  to  illustrate  a point,  or  he  related  a point  he  was  making  to  an  auto  owned 
by  one  of  the  students,  the  students  appeared  highly  interested  in  the  subject  and  all  attended  to  the 
instruction  As  a further  illustration  of  the  strong  goal  orientation,  th»re  were  frequent  inquiries  as  to 
whether  a subject  would  be  on  the  test.  There  was,  however,  a general  lack  of  attentiveness  during  the  more 
abstract  explanations  of  operating  principles.  This  led  the  observers  to  believe  that  the  students  did  not 
really  care  how  something  worked,  that  they  just  wanted  to  know  how  to  fix  it.  This  observation  is  further 
reinforced  by  student  behavior  on  the  topic  of  maintenance  manuals  or  technical  orders.  Here  the  students 
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had  to  learn  how  to  use  an  index  to  find  the  proper  maintenance  manual,  then  had  to  find  the  proper 
chapter,  and  finally  had  to  determine  if  it  was  up-to-date.  Students  were  observed  not  to  take  this  exercise 
very  seriously.  There  was  much  disorder,  constant  repetition  of  instructions  was  necessary,  and  several 
students  still  were  unable  to  pass  the  end-of-lesson  test  on  this  subject.  Very  much  the  same  thing  happened 
with  the  lesson  on  security. 

These  observations  pointed  up  a weakness  and  a strength  of  Chanute’s  lessons.  Firet,  the  lessons  were 
unable  to  sense  the  needs  of  an  individual  or  group  at  the  level  that  an  instructor  could,  and  then  be 
responsive  to  these  needs.  Nevertheless,  PLATO  had  a definite  advantage  in  the  presentation  of  difficult  and 
potentially  unattractive  instructional  sequences;  such  as  technical  orders  and  security.  The  power  of  the 
system  to  control  the  instruction,  repeat  untiringly,  and  drill  to  insure  mastery  was  invaluable.  Moreover, 
the  consistency  of  CAI  insured  that  each  student  received  instructions  free  from  problems  caused  by 
instructor  variability  in  presentation. 

Instructional  Styles  The  third  area  of  observation  that  could  be  compared  across  media  was  that  of 
instructional  styles.  These  styles  related  to  the  manner  in  which  the  content  was  delivered. 

In  the  conventional  classroom,  perhaps  the  most  noticeable  observation  was  that  there  were  distinct 
differences  in  the  personalities  and  teaching  “techniques”  between  instructors.  These  “techniques”  were 
not  limited  to  the  clarity  of  instruction,  but  included  the  way  the  instructor  interacted  with  the  students 
both  as  a teacher  and  as  a disciplinarian.  Because  the  students  got  a new  instructor  every  2 weeks,  there  was 
always  a period  of  uneasiness  at  each  transition,  while  the  students  “felt  out”  their  new  teacher.  During  the 
transition  there  was  usually  much  less  tendency  for  students  to  ask  questions,  but  the  attention  level 
seemed  higher.  Not  enough  observations  were  made,  however,  to  determine  if  time  taken  to  adjust 
decreased  as  the  number  of  transitions  increased. 

Just  as  students  had  to  adjust  to  new  teaching  styles  of  instructors,  it  was  noted  that  they  also  had  to 
adjust  to  different  teaching  styles  across  PLATO  lessons.  In  these  CAI  lessons,  there  were  various  strategies 
employed  in  no  particularly  consistent  pattern.  Some  programs  were  simple  and  linear;  some  involved 
branching  to  new  material  on  a mandatory  or  optional  basis;  some  included  remedial  loops  that  sent  a 
student  back  to  repeat  old  material  if  he  failed  a test  item;  and  some  employed  choice  pages  that  allowed 
the  student  to  choose  his  own  sequence  of  material.  In  the  majority  of  cases,  students  had  not  bee" 
previously  exposed  to  these  kinds  of  instructional  strategies  in  programmed  instruction.  Now  suddenly, 
they  found  themselves  in  a position  where  they  needed  to  make  decisions  about  how  and  what  to  study,  if 
only  on  a small  scale.  It  was  interesting  to  note,  though,  that  once  students  learned  to  manipulate  the 
PLATO  system,  they  quickly  adopted  methods  of  “beating  the  system.”  for  example,  several  students 
found  ways  of  going  directly  to  an  end-of-lesson  test  (Master  Validation  Examination  or  MVE)  without 
going  through  the  entire  lesson.  Some  found  that  if  they  could  pass  a pretest,  they  could  avoid  taking  the 
lesson.  Therefore,  they  concentrated  upon  taking  the  pretest  over  and  over  again,  hoping  to  pass  it  by 
chance  alone  and  not  having  to  go  through  the  lesson.  Needless  to  say,  it  was  necessary  to  redesign  the  s 

lesson  programs  to  prevent  these  kinds  of  behaviors  from  continuing. 

Disivssion.  A basic  conclusion  to  be  drawn  concerning  instructional  style  was  that  control  of  the 
instructional  material  must  be  handled  very  carefully.  It  should  not  be  assumed  that  students  are  motivated 
to  do  the  best  job  they  can  in  the  time  available,  for  in  any  large  group  of  students  there  will  always  be 
some  who  take  the  line  of  least  resistance.  But  this  does  not  mean  that  student  control  should  be 
completely  eliminated,  for  highly  motivated  and  able  students  can  profit  from  both  pretests,  which  allow 
them  to  avoid  unneessary  instruction  and  from  other  options  they  can  use  to  tailor  the  instruction  to  their 
needs.  Thus,  a corollary  can  be  drawn  such  that  any  instructional  material  which  can  be  circumvented 
eventually  will  be.  and  for  every  decision  a student  must  make,  no  matter  how  simple  it  appears,  there  is  a 
finite  probability  that  he  will  make  the  wrong  decision. 

Conclusion.  In  summary,  implementing  PLATO  was  observed  to  have  advantages  and  disadvantages  in  * 

the  area  of  (a)  conduct  of  training,  (b)  student  interaction,  (c)  instructional  content,  and  (d)  instructional 
styles.  In  the  conduct  of  training,  PLATO  was  seen  to  have  mostly  disadvantages.  It  was  available  only  part 
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of  the  training  day  and  there  were  times  when  EiCS  limitations  permitted  use  of  only  a few  terminals  or 
lessons.  The  fact  that  PLATO  was  not  an  Air  Force  resource  was  pointed  out  as  a major  drawback.  For 
student  interaction,  PLATO  supplied  benefits  in  assuring  comprehensive  and  consistent  student 
questioning,  but  suffered  in  adaptiveness  and  gcnerali/ability.  Instructional  content  was  improved  by 
PLATO’s  ability  to  provide  consistent  and  interesting  lessons.  It  was  hindered  somewhat,  on  the  other 
hand,  by  an  inability  to  be  responsive  or  appropriate  at  unpredictable  points  in  teaching.  Finally,  with 
respect  to  instructional  style,  consistency  of  PLATO  was  an  advantage,  but  the  complexity  of  some  lesson 
designs  and  decision  requirements  was  a disadvantage. 

Two  basic  conclusions  were  reached.  First,  PLATO  vas  not  found  to  have  a major  adverse  impact 
that  precluded  its  potentially  effective  use  in  technical  training.  Second,  refinement  of  lesson  material  and 
improved  scheduling  of  terminals  and  lessons  could  reduce  the  observed  negative  impact  on  conduct  of 
training. 

Impact  on  Existing  Air  Force  Regulations  and  Guidance 

It  was  thought  that  implementation  of  PLATO  would  require  a great  many  changes  in  training 
methodology  which  consequently  would  need  to  be  reflected  through  changes  in  numerous  Air  Force 
manuals,  regulations,  and  pamphlets.  However,  such  an  expectation  was  not  fulfilled  with  the  utilization  of 
the  PLATO  IV  system  at  Chanute,  and  probably  would  not  be  fulfilled  at  other  Air  Force  installations  that 
may  eventually  become  involved  with  CAI/CM1  systems. 

Tire  basic  manuals  and  pamphlet  guiding  Instructional  Systems  Development  arc  AFM  50-2  and  AFP 
50-58. 

AFM  50-2  Instructional  Systems  Development , contains  the  concepts  and  principles  to  begin 
structuring  the  systemization  process. 

AFP  50-58,  Handbook  for  Designers  of  Instructional  Systems,  outlines  the  application  of  philosophy, 
theory,  concepts,  and  principles  in  AFM  50-2. 

Together,  these  three  sources  form  the  reference  literature  to  analyze,  plan,  develop,  validate,  and 
evaluate  an  instructional  system.  Because  innovation  is  a hallmark  of  the  entire  ISD  process,  the  use  of  CA1 
and  CMI  was  anticipated  and  included  as  a possible  alternative  in  this  reference  material. 

Because  the  use  of  CAI/CMI  is  already  included  as  policy  in  the  major  reference  literature  for  ISD,  and 
because  ISD  policy  is  the  governing  policy  in  Air  Force  technical  training,  regulations  written  for  ATC 
usage  also  include  provisions  for  CAI/CMI  implementation.  Although  not  specifically  mentioning  these 
concepts,  the  regulations  do  reflect  capabilities  for  educational  innovation.  The  ATC  50-  and  52-series 
regulations  have  the  greatest  applicability  to  PLATO  (as  used  in  the  current  context).  The  following 
portions  of  those  regulations  deal  with  innovation  and  provide  for  CAI/CMI  implementation  in  technical 
training: 

ATCR  52-3,  (cl)5m,  Measurement,  provides  for  innovations  in  test  and  analysis  procedures. 

ATCR  52-6,  2a,  Course  Charts,  cites  AFM  50-2  for  guidelines,  which  provides  for  innovations. 

ATCR  52-7  2a,  Plans  of  Instruction  and  Lesson  Plans  cites  AFM  50-2  for  guidelines. 

ATCR  52-18,  2f,  Training  Materials,  indicates  that  one  may  deviate  from  ATC  approved  course 
control  documents  to  determine  the  feasibility  of  proposed  course  changes.  This  regulation  is  normally 
cited  when  deviating  from  course  control  documents  to  validate  an  ISD  product  and  appears  quite  flexible. 

ATCR  50-5  1,  4d,  Instructional  Systems  Development  indicates  that  innovations  will  be  utilized  to 
develop  cost  effective  programs.  AFM  50-2  is  cited  and  requests  *o  deviate  arc  included  as  policy. 

To  summarize  the  findings,  regulations,  pamphlets,  and  manuals  utilized  at  the  technical  training  level 
were  examined  and  found  to  make  provisions  for  CAI/CMI  innovation  and  change  in  measurement,  course 
charts,  plans  of  instruction,  and  systems  development  through  normal  training  deviation  procedures. 
Finally,  the  PLATO  IV/ISD  process  was  compared  to  a typical  ISD  effort.  After  the  similarities  and 
differences  were  explored,  it  was  discerned  that  very  little  that  was  abnormal  occurred  with  the  PLATO  IV 
effort. 
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In  conclusion,  further  PLATO  efforts  at  other  Air  Force  installations,  if  conducted  in  conjunction 
with  a systematic  in-depth  Instructional  System  Development  effort,  should  anticipate  few  difficulties  in 
coping  with  existing  regulations. 

New  Training  Capabilities 

Because  of  the  relatively  brief  period  available  for  experimental  applications  of  PLATO,  it  is 
impossible  to  ,-rcsent  concrete  evidence  of  training  capabilities  provided  by  PLATO  which  cannot  be 
provided  by  traditional/conventional  media.  The  primary  reason  for  this  lack  of  information  is  that  pressure 
to  get  operational  training  underway  as  soon  as  possible  prevented  experimentation  with  teaching  new  skills 
through  PLATO.  In  order  to  group-pace  students,  it  was  necessary  to  devote  full  attention  to 
individualizing  the  classroom  instruction.  Tints,  PLATO-based  laboratory  training,  where  the  unique 
capabilities  of  PLATO  could  most  dramatically  be  demonstrated,  was  not  attempted.  However,  based  upon 
work  at  other  locations  and  assessment  of  the  successful  applications  of  animation  and  answer-judging 
capabilities  employed  in  this  effort,  it  seems  evident  that  PLATO  could  be  used  to  teach  laboratory  skills 
now  being  taught  through  conventional  instructor  demonstration  and  practice. 

Two  areas  in  particular  suggest  themselves  as  potentially  cost-effective  applications  of  PLATO  in 
technical  training.  These  areas  are  diagnosis/troublcshooting  and  performance  testing. 

Diagnosis  and  troubleshooting  are  important  skills  for  a mechanic  to  possess,  yet  they  are  uniquely 
difficult  to  teach.  First,  in  order  to  teach  diagnostics,  it  is  necessary  to  have  faults  to  diagnose.  Training 
equipment  that  is  faulty  must  be  simulated  by  special  training  devices.  Consequently,  it  is  necessary  to 
teach  diagnostics  through  expensive  equipment.  With  PLATO,  it  is  possible  to  present  low  fidelity 
simulations  of  engine  malfunctions.  Furthermore,  the  interactive  capatilitics  of  the  system  allow  students  to 
receive  immediate  in-depth  feedback  concerning  the  accuracy  of  diagnosis.  In  addition,  capabilities  for 
randomly  generating  simulated  faults  enable  students  to  experience  a wide  variety  of  faults  which  would 
not  be  feasible  with  less  dynamic  or  expensive  media. 

Furthermore,  the  strategy  of  diagnosis  or  troubleshooting  could  be  taught  in  a dynamic  interactive 
mode  which  closely  conformed  to  behaviors  required  in  the  field.  Thus,  the  capabilities  of  PLATO  could 
enable  the  teaching  of  a difficult  skill  to  an  acceptable  level  within  the  time  restrictions  of  technical 
training  something  that  cannot  be  done  through  traditional  media. 

The  second  area,  where  PLATO  could  provide  significant  new  capabilities  to  technical  training,  is  in 
the  area  of  performance  measurement.  At  present,  the  only  quantitative  measure  of  student  performance  is 
provided  by  multiple-choice  or  true/false  appraisals  and  block  examinations.  These  measures  have  only  an 
indirect  relationship  to  a student’s  ability  to  actually  perform  the  task  activities.  The  only  direct  evidence 
of  task  performance  comes  from  the  highly  subjective  pass/fail  judgment  of  the  instructor  in  the  laboratory. 
With  PLATO,  however,  it  would  be  possible  to  present  the  procedural  aspects  of  various  tasks  and  to 
measure  how  well  students  can  follow  the  TO  in  accomplishing  the  tasks.  While  still  indirect,  this  procedure 
would  be  much  more  faithful  to  actual  task  performance  and  would  give  a measure  of  each  student's  ability 
to  perform.  Some  laboratory  tasks  are  such  that  students  perform  them  in  groups  or  only  see  a 
demonstration  which  allows  some  students  to  go  through  training  without  actually  having  reached  a 
specified  level  of  proficiency  on  all  tasks. 

Tlte  importance  of  this  capability  is  that  it  permits  a direct  estimate  of  the  transfer  of  classroom 
instruction  to  performance  on  actual  tasks.  In  this  manner,  it  would  be  possible  to  avoid  generalized  “nice 
to  know"  information  and  to  tie  instruction  in  basic  principles  and  fundamentals  directly  to  tire  tasks  to  be 
performed. 

Incorporation  of  these  increased  training  capabilities  could,  if  carefully  done,  significantly  improve 
the  efficiency  of  vehicle  maintenance  training.  Whether  or  not  this  increase  in  efficiency  is  cost-effective, 
however,  depends  upon  a great  many  factors  ranging  from  design  and  coding  of  the  software  to 
management  of  student  and  instructor  resources. 
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Overall  Summary  and  Conclusions 

The  impact  of  PLATO  on  attitudes  of  students  was  generally  positive.  Students  in  the  PLATO 
training  conditons  had  more  positive  attitudes  toward  training  than  their  non-PLATO  counterparts.  While 
attitudes  of  all  students  tended  to  become  less  positive  over  the  four  blocks  of  training  involved  in  the 
service  test,  attitudes  of  PLATO  students  tended  to  decline  more  slowly  than  those  of  non-PLATO 
students.  Responses  to  specific  attributes  of  PLATO  indicated  that  students  enjoyed  working  with  the 
system  and  found  it  to  be  an  effective  means  of  teaching. 

Instructors  reported  initially  positive  attitudes  toward  PLATO,  but  after  several  months  these 
attitudes  declined  significantly.  It  was  suggested  that  the  reason  for  this  decline  in  attitude  was  most  likely 
some  dissatisfaction  with  the  less  direct  role  of  instructors  in  PLATO  instruction. 

Daily  training  activities  were  most  severely  impacted  by  restricted  operating  hours  and  occasional 
ECS  shortage.  These  problems  required  that  unusual  management  procedures  be  implemented  to 
accommodate  training  interruptions.  Other  advantages  and  disadvantages  were  pointed  out  in  the  areas  of 
student  interaction  in  training,  instructional  content,  and  instructional  style.  The  overall  conclusion  was 
that  PLATO  had  no  significantly  adverse  impact,  which  would  preclude  its  potentially  cost-effective  use  in 
technical  training. 

Examining  the  impact  of  PLATO  on  existing  ATC  regulations,  governing  conduct  of  training,  revealed 
that  no  major  changes  in  these  documents  were  necessary  in  order  to  incorporate  the  new  system. 

While  it  was  not  possible  to  experiment  with  new  training  capabilities,  a discussion  of  potential  new 
capabilities  was  presented.  This  discussion  pointed  out  that  two  areas,  troubleshooting  and  performance 
testing,  arc  within  the  capability  of  the  PLATO  system  and  could  significantly  enhance  cost-effectiveness 
of  vehicle  maintenance  training.  Additional  emphasis  on  use  of  PLATO  for  these  two  areas  is  strongly 
encouraged. 

In  conclusion,  the  overall  impact  of  PLATO  has  been  more  positive  than  negative.  There  is  no  reason 
to  believe  that  PLATO  cannot  prove  to  be  an  acceptable  and  useful  training  medium. 


VI.  INSTRUCTIONAL  MATERIALS  DEVELOPMENT 

The  focal  points  of  this  portion  of  the  evaluation  are  the  process  used  to  develop  CAI  training 
materials,  the  factors  influencing  this  process,  an  evaluative  description  of  the  validated  lessons,  and 
findings  from  the  validation  of  the  lessons.  The  following  observations  pertain  to  Phase  II,  or  from  the  time 
the  ISD  and  PLATO  efforts  merged,  until  January  1976. 

Method 

Observational  and  interview  techniques  were  used  to  gather  the  information  for  this  evaluation  area. 
Instrumented  interviews  of  project  personnel  were  conducted  in  November  and  December,  1974.  These 
interviews  provided  the  basis  for  describing  the  lesson  development  paradigm  and  presenting  the  factors 
which  influenced  lesson  development  activities.  Descriptions  of  lessons  and  their  validation  characteristics 
relied  upon  reviews  performed  by  the  evaluation  team  and  members  of  the  CERL  staff. 

Results  and  Discussion 

Process  Description.  One  of  the  principal  project  objectives  was  the  derivation  of  a process  to  produce 
effective  CAI  materials.  The  process  had  to  integrate  the  concept  of  author/instructor,  the  needs  of  a 
military  technical  training  setting,  the  capabilities  and  limitations  of  the  PLATO  system,  and  other  diverse 
variables.  During  Phase  I,  experience  concerning  these  variables  accrued,  yet  no  gcncralizable,  well-defined 
process  emerged.  The  period  saw  authors  developing  lessons  with  varied  methods.  The  efficiency  and  utility 


of  these  methods  were  unknown  and  questionable.  During  Phase  11,  project  personnel  developed  a process 
which  integrated  concepts,  needs,  and  experience.  Efficiency  was  gained  and  the  basic  model  may  be 
generalizable  to  oilier  military  and  civilian  applications  of  CA1. 

The  Nature  of  Technical  Training.  The  development  of  PLATO  Courseware  was  defined  in  part  by 
the  nature  of  Air  Force  technical  training.  In  general,  the  training  received  for  qualification  in  a career  field 
is  accomplished  in  two  steps.  First,  the  airman  is  sent  to  resident  technical  school  where  basic  career  field 
knowledges  and  skills  arc  learned.  After  graduation,  the  student  undertakes  a first  assignment  at  an 
apprentice  level  where  the  career  field  knowledges  and  skills  are  refined  by  a Career  Development  Course 
(CDC),  and  on-the-job  training  (OJT).  Career  field  tasks  are  periodically  analyzed  so  that  the  scope  and 
depth  of  training  is  commensurate  with  the  needs  of  the  career  field.  The  technical  school  graduate,  an 
apprentice  mechanic,  receives  training  on  a specific  set  of  skills,  and  attains  a certain  level  of  proficiency 
that  allows  the  airman  to  step  into  the  next  phase  of  training.  To  achieve  this  synchronization,  a 
standardized,  controlled  course  curriculum  is  required. 

An  important  distinction  exists  between  training  and  education.  In  educating  a person,  the  teacher 
provides  a series  of  learning  experiences  to  aid  the  student  in  concept  and  skill  attainment.  The  scope  and 
depth  of  education  are  highly  flexible,  to  accommodate  a multiplicity  of  variables.  Within  the  context  of 
technical  training,  the  desired  scope  and  depth  of  learning  outcomes  are  fairly  well  defined  for  the  various 
portions  of  student  training.  As  a consequence.  Air  Force  technical  training  cannot,  and  should  not.  have 
latitude,  for  the  real  cost  of  resources  would  exceed  the  benefits  of  a generalized  training  experience. 

The  concept  of  sufficient  training  at  minimum  cost  (lean  training)  is  highly  relevant  to  the  Air  Force. 
The  student  who  enters  the  second  phase  of  training  should  have  only  those  basic  skills  and  knowledges 
that  arc  required  for  success  in  OJT.  Overtraining,  in  a sense,  is  waste  and  waste  is  costly.  It  increases 
training  time  and  decreases  available  productive  time. 

This  task-specific  nature  of  technical  training  has  a direct  impact  upon  curriculum  development.  The 
ISD  team  must  use  a lean  approach  in  developing  an  instructional  system  wherein  courseware,  such  as 
lessons,  is  written  with  little  elaboration  (i.e.,  examples,  detailed  explanations)  and  in  large  instructional 
steps.  The  courseware  is  then  put  into  successive  tryout  and  validation  trials  where  it  is  refined  and 
augmented  until  90  percent  of  the  students  meet  all  standards  specified  in  the  instructional  objectives.  In 
this  way,  the  requirement  for  sufficient  training  is  met  while  not  overtraining  or  providing  superfluous 
information. 

CAI  Lesson  Development 

The  Model.  The  procedural  model,  described  in  this  section,  is  a generalized  composite  of  the  process 
actually  used.  In  other  words,  like  all  models,  deviations  frequently  were  found  due  to  realistic  constraints, 
or  the  individual  differences  of  the  people  who  used  it.  However,  the  basic  paradigm  held  for  the  majority 
of  lessons. 

A team  approach  was  adopted  for  use  with  the  lesson  development  process  because  prior  experience, 
and  the  conditions  prevalent  at  the  time,  indicated  this  to  be  the  best  alternative.  One  beneficial  effect  of  a 
team  effort  was  the  reduction  of  ego  involvement  and  the  promotion  of  objectivity.  Also,  the  specialization 
of  roles  allowed  for  greater  efficiency  since  personnel  accomplished  the  tasks  they  were  best  suited  to  do. 
But  most  important,  the  possibility  of  initial  high  quality  products  increased,  owing  to  the  variety  of  talents 
and  ideas  available,  along  with  the  mutual  monitorship  of  team  member  activities. 

The  lesson  development  teams  at  Chanute  consisted  of  four  positions:  an  author  who  had  the  primary 
responsibility  for  generating  the  lesson,  a subject-matter  expert  (SME)  who  provided  technical  expertise  for 
the  author,  an  instructional  programmer  (IP)  who  assisted  the  lesson  author  with  programming  principles, 
and  a computer  coder  who  helped  the  author  in  coding  a lesson,  or  in  some  cases  coded  the  entire  lesson. 
The  central  figure  was  the  author,  a necessity,  since  author  skills  had  been  developed  on  a variety  of 
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dimensions  by  this  phase  of  the  service  test.  From  July  1974  to  January  1975,  two  such  teams  were  used  to 
develop  lessons.  Since  the  author  did  almost  all  of  the  work  on  a lesson,  each  team  consisted  of  two  or 
three  authors  by  an  SME,  IP,  and  coder. 

In  presenting  the  model,  the  framework  of  the  model’s  procedural  elements  will  be  examined  first. 
The  main  procedural  elements  are  designated  as  steps  even  though  they  were  not  clearly  delineated  in  this 
manner  by  project  personnel.  Following  the  description  of  the  model  is  an  assessment  of  the  procedural 
aspects  in  relation  to  current  ISD  methodology. 

Step  1 : Lesson  Selection 


CAI  lessons  were  developed  as  a part  of  the  ISD  model’s  Step  4 — "Ran.  Develop,  and  Validate 
Instruction”  (AFM  50-2).  Lesson  authors  had  available  to  them  the  course  objectives,  criterion  tests  or 
master  validation  examinations  (MVE),  and  teaching  points.  These  were  prepared  during  Step  Three  of  the 
ISD  Model  (Figure  11).  As  the  first  lesson  preparation  activity,  the  author  selected  an  objective  and  its 
associated  MVE  and  teaching  points  from  the  pool  of  cognitive  objectives  that  the  ISD  teams  had  targeted 
for  PLATO.  The  author  then  initiated  a lesson  documentation  folder  where  team  members  recorded  lesson 
development  notes  and  time  expenditures,  and  stored  lesson  control  documents,  hard  copies  of  the  lesson, 
and  the  TUTOR  coding. 


CONSTRAINTS 


CONSTRAINTS 


LEGEND  CURRICULUM  LOOP  ■ 

FEEDBACK  AND  INTERACTION  LOOP  « » 

Figure  II.  Model  for  instructional  system  development  (AFM  50-2). 

Step  2:  Research 


To  become  familiar  with  the  subject  matter,  the  author  researched  the  available  information  related 
to  the  selection.  Consulting  with  the  team’s  IP  and  SME,  the  author  examined  the  adequacy  of  and 
relationships  between  objective.  MVE,  and  teaching  points.  If  acceptable,  the  author  proceeded  with  the 
next  step;  if  not,  then  changes  were  made  and  coordinated  with  the  Special  Purpose  Vehicle  Repairman 
course  management. 


Step  3:  Lesson  Hanning 


The  author  considered  an  approach  to  subject  matter  presentation  via  the  PLATO  terminal.  The 
author  established  the  overall  lesson  strategy,  the  peripheral  media  anticipated  (touch  panel,  microfiche. 


56 


audio),  interaction  modes,  and  the  sequence  of  information.  A programming  plan  was  then  prepared  for 
review  by  the  team  IP  and  SME.  In  some  cases,  a coder  review  was  included  if,  for  example,  internal  lesson 
routines  required  checking  for  feasibility  of  coding. 

Step  4:  Off-Line  Lesson  Preparation 


The  author  then  initiated  the  procurement  of  the  lesson  training  aids  such  as  slides,  drawings,  or 
adjunct  components.  The  textual  portions  of  the  lessons  were  written  or  outlined,  diagrams  sketched,  and 
the  lesson  interactive  strategies  produced,  hollowing  these  activitics.the  lesson  was  coded. 

Step  S:  On-Line  Coding 


The  authors  had  the  option  of  doing  all.  part,  or  none  of  the  on-line  coding  for  a particular  lesson 
Typically,  they  selected  portions  of  their  lesson  that  were  the  most  difficult  or  the  most  tedious  for 
assistance  from  the  team  coding  specialist.  The  lesson  was  coded  into  its  initial  format. 

Step  (•  Pre-Tryout  Review 

Before  the  lesson  was  tested  on  small  groups  of  students,  it  was  desirable  for  the  team  to  check  the 
technical,  programming,  and  grammatical  accuracy  of  the  lesson,  as  well  as  the  acceptability  of  the  TUTOR 
coding.  In  the  event  the  lesson  required  modificiation.  the  team  member  requesting  it  conferred  with  the 
author  to  assess  the  extent  of  change.  The  author  then  adjusted  the  lesson. 

Step  7:  Student  Tryout 


In  trying  out  the  lesson  with  small  groups  of  students,  the  author  hoped  to  obtain  information 
concerning  the  effectiveness  of  the  lesson  as  a whole,  and  of  its  individual  segments.  Major  changes  to  the 
lesson  were  generally  not  made  at  this  stage,  since  the  data  represented  the  results  of  a small  sample  of 
students  (1  to  6 students,  approximately  three  tryout  attempts).  In  addition,  an  author  could  request 
course  instructors  to  review  the  lesson  as  an  added  source  of  feedback  and  as  a method  to  keep  course 
personnel  involved  with  courseware  development. 

Step  8:  Validation 


Tire  lesson  was  then  tested  in  context  with  all  other  new  courseware  until  90  percent  of  the  students 
achieved  the  standard  of  the  objective  on  their  firs!  attempt  at  the  MVE.  Approximately  30  students,  or 
three  consecutive  classes,  comprised  the  validation  sample.  After  the  validation  attempt  of  each  class,  the 
author  examined  the  summary  data  for  the  lesson  to  determine  if  validation  was  possible.  In  the  event  the 
lesson  required  revision,  several  data  sources  for  diagnosing  specific  lesson  deficiencies  were  available.  These 
included  an  MVE  item  analysis,  lesson  time  parameters,  and  internal  lesson  responses. 

Step  9:  Finalization 


As  the  final  step  in  the  preparation  of  a CAI  lesson,  the  team  instructional  programmer  performed  a 
grammatical  edit  since  several  portions  of  the  lesson  had  probably  been  rewritten  during  the  validation  step. 
The  lesson  was  then  presented  to  the  course  management  personnel  for  an  acceptance  check. 

Assessment  of  the  Model.  As  with  most  procedural  models,  the  CAI  lesson  development  process 
represented  an  ideal  set  of  procedures.  There  was  no  formal  written  procedure,  only  an  understanding  in 
the  minds  of  the  lesson  development  teams  and  the  ISD  management.  Several  times  this  collective 
comprehension  was  extremely  inadequate,  a result  of  poor  communication  within  teams  and  between  teams 
and  management. 
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The  model  was  not  used  to  produce  administration  and  management  routines.  When  an  on-line  testing 
and  test  item  analysis  routine  or  a student  router  was  needed,  a coder  would  work  from  a set  of 

specifications  coordinated  with  the  ISD  team.  Frequently  these  specifications  were  verbal,  making  use  of 

the  expertise  of  the  coder.  In  one  case,  an  on-line  version  of  the  course  plan  of  instruction  (POI)  was 

produced  by  an  ISD  team  IP  who  had  taken  interest  in  coding  and  felt  that  an  on-line  version  would 

expedite  preparation. 

Even  though  some  limitation  exist,  the  procedural  model  is  compatible  with  present  ISD 
methodologies  for  the  development  of  training  materials  within  the  Air  Force  technical  training.  The  ISD 
team  required  a lesson  development  subsystem  whose  output  was  commensurate  with  the  primary  goal  of 
developing  an  instructional  system.  Accordingly,  lesson  development  for  CA1  was  an  adaptation  of  the  basic 
model  described  in  AFP  50-58,  Vol.  4. 

The  important  contrasts  between  the  Chanute  CA1  lesson  preparation  model  and  off-line  media 
development  models  (with  the  possible  exception  of  some  audiovisuals)  fell  under  the  major  categories  of 
process  control,  ad  hoc  functions,  and  automation  impacts.  In  relation  to  process  control,  several  additional 
check  points  were  included  in  the  CAI  lesson  model  for  review  of  the  lesson  progress,  a desirable  feature  in 
view  of  the  time  investment  in  coding  erroneous  or  irrelevant  material.  Most  tasks  which  the  IP  and  SME 
performed  were  control  functions,  not  material  preparation.  The  provision  of  teaching  points  with 
objectives  and  MVEs  was  not  a normal  procedure  for  this  ISD  team,  but  it  provided  an  additional  control 
measure. 

Die  design  of  the  model  was  influenced  by  the  number-  and  background-  of  available  personnel.  The 
coding  function  obviously  influenced  the  nature  of  the  model  and  its  deviation  from  the  process  used  in  the 
development  of  other  programmed  materials.  For  example,  it  took  an  SME  about  40  hours  to  develop  a 
1-hour  paper-and-pcncil  programmed  text  with  moderate  interaction  with  an  IP.  It  took  a PLATO  author 
approximately  3 to  4 times  that  to  produce  the  same  validated  instructional  hour  with  substantial 
interaction  with  his  teammates.  Moreover,  lesson  edits  were  complicated  by  the  coding  feature,  since  the  IP 
had  to  inform  the  author  a change  was  required ; then  verify  its  consumation. 

At  this  point,  one  important  advantage  of  the  PLATO  system  should  be  mentioned.  The  use  of 
on-line  data  collection  routines,  adapted  from  those  available  from  CERL,  took  much  of  the  drudgery  from 
the  validation  task.  A summary  of  the  most  important  data  was  almost  instantaneously  available  after  a 
validation  attempt.  Likewise,  the  tedious  textual  lesson  revision  activities  which  normally  require  days, 
weeks,  or  months  (depending  on  the  extent  of  revision),  were  typically  done  in  a few  hours,  or  at  most, 
days.  In  these  ways,  automation  expedited  the  lesson  revision  process. 

There  are  a lew  serious  limitations  to  this  procedural  model.  Die  foremost  is  its  time-consuming 
nature.  Die  team  coordination  and  control  activities  (along  with  the  inefficient  use  of  team  specialists) 
contribute  heavily  to  this  limitation.  The  model  also  required  an  excessive  dependence  on  the  author,  who 
did  most  of  the  development  work  and  utilized  other  team  members  as  resources.  Consequently,  the 
designation  “team  approach"  was  a nominal  one. 

The  assumption  was  made  that  principles  for  effective  media  utilization  were  known.  Die  lesson 
selection  and  lesson  planning  stages  required  that  the  team  members  would  make  appropriate  decisions, 
based  on  their  knowledge  of  CAI  and  its  application.  With  few  such  principles  established,  the  efficient 
execution  of  these  stages  was  impossible. 

Despite  these  limitations,  the  model  was  chosen  as  the  best  available  for  Chanute  during  June  1974.  It 
evolved  from  the  conditions  present  at  the  time,  to  satisfy  the  need  for  a structured  process  that  could 
effectively  and  efficiently  turn  out  lessons  commensurate  with  Air  Force  technical  training  philosophy. 

Appendix  A contains  a lesson  development  scenario.  The  intent  of  this  scenario  is  to  provide  the 
interested  reader  with  a detailed  description  of  lesson  development  activities,  associated  problems,  and 
other  factors  influencing  their  design. 
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Courseware  Development  Issues  and  Constraints.  Many  of  the  PLATO  courseware  development 
problems  during  the  first  two  project  phases  were  site-specific.  For  example,  during  the  20-month  Phase  I, 
the  PLATO  system  was  subject  to  rapid  growth  and  erratic  behavior.  Project  personnel  were  struggling  with 
orientation  and  management  problems,  as  well  as  the  lengthy  military  and  civilian  author  acquisition  and 
training  times.  These  important  influences  for  the  most  part  inhibited  lesson  preparation  activities,  and 
could  be  classified  as  issues  and  constraints  faced  by  project  personnel.  After  orienting  the  reader  to  the 
prior  lesson  development  efforts  of  Phase  I,  the  issues  and  constraints  faced  during  the  three  phases  are 
discussed  under  the  headings  Project  Management,  PLATO  System  Hardware/Software,  Courseware 
Considerations,  and  Authoring. 

Prior  Lesson  Development  efforts.  During  the  initial  phase  of  the  project,  each  author  performed  his 
lesson  development  roles  in  an  independent  fashion  following  the  paradigm  observed  at  CERL  during 
TUTOR  training.  Efforts  to  produce  lessons  in  a quasi-team  fashion  still  placed  prime  responsibility  on  the 
author  and  consequently  were  short-lived.  Under  this  management  philosophy,  little  accountability  existed 
and  the  standards  for  quality  control  of  lessons  were  set  by  their  respective  authors.  This  model  of  lesson 
development  was  termed  the  Autonomous  Author  Approach  (Dallman,  1974)  and  was  suited  to  the  natures 
of  the  original  eight  authors.  These  authors  were  primarily  selected  on  the  basis  of  youth  and  education, 
with  the  assumption  that  a homogeneous  group  (with  the  desired  characteristics  of  low  resistance  to 
innovation  and  rapid  skill  acquisition)  would  result.  A trade-off  had  to  be  made,  and  instructor  experience 
and  subject  matter  expertise  were  sacrificed.  The  nature  and  background  of  the  original  eight  authors  is 
further  documented  by  Green  (1973). 

Throughout  the  2-year  autonomous  author  era,  a variety  of  methods  was  attempted  to  upgrade 
author  subject  matter  knowledge  and  authoring  skills  such  as  instructional  programming  and  coding.  Of 
these,  the  coding  skill  required  the  lion’s  share  of  an  author’s  attention  to  learn.  This  condition  was,  in  part, 
due  to  the  fascination  with  the  ability  to  manipulate  a complex  machine,  and  be  almost  instantaneously 
intrinsically  rewarded  with  the  results  of  invested  efforts. 

Some  serious  constraints  hindered  the  attainment  of  project  goals.  The  Chanute  test  personnel 
(project  management,  authors)  had  a basic  inability  to  recognize  and/or  respond  to  the  needs  of  an 
extensive  curriculum  development  project.  Project  management  had  little  experience  with  operational 
technical  training  and  even  less  with  curriculum  development.  Consequently,  placing  trust  in  the  project 
personnel  through  a laissez-faire  management  scheme  seemed  to  be  appropriate.  It  had  been  assumed  that 
ne  author  group  would  obtain  whatever  it  required  to  get  a PLATO  version  of  the  target  course 
operational.  CERL  had  adhered  to  the  philosophy  of  individual  authors  developing  CAI  lessons,  and  this 
reinforced  the  decision.  Unfortunately,  the  primary  assumption  upon  which  the  philosophy  was  dependent, 
namely,  that  an  author  is  knowledgeable  in  both  subject  matter  and  educational  practices,  was  overlooked. 
Chanute’s  authors  were  not  subject  matter  experts,  nor  did  they  have  substantive  experience  with 
educational  practices.  An  emphasis  on  research  within  the  confines  of  an  operationally  oriented  technical 
training  environment  created  a condition  of  passive  interest  in  higher  management  and,  also,  a feeling  that 
the  project  would  be  short-lived. 

Other  constraints  present  during  this  period  limited  productivity.  The  evolving  PLATO  system  had 
relatively  poor  reliability  in  software  and  hardware.  Inexperienced  personnel  failed  to  recognize  the 
philosophy  and  needs  of  Air  Force  technical  training,  and  their  assistance  helped  to  perpetuate  the  existing 
lesson  development  environment.  In  addition,  hardware  delivery  date  slippages,  poor  communication,  and 
unstable  relations  with  the  host  course  had  a detrimental  effect  on  development  activities. 

The  consequences  of  an  autonomous  author  approach  within  the  project  setting  were  readily 
apparent  to  the  attendees  of  the  May  1974  project  review,  which  signaled  the  end  of  Phase  I.  The 
curriculum  was  fragmented  with  little  continuity  between  lessons.  Lessons  did  not  relate  to  the  philosophy 
of  Air  Force  technical  training,  and  followed  few  of  the  accepted  programmed  instruction  principles.  A 
change  to  experienced  management  was  deemed  the  best  solution. 
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Project  Management.  The  transition  from  the  research  atmosphere  of  Phase  I to  the  rigid  production 
environment  of  Phase  II  was  abrupt.  The  ISD  team  did  not  have  the  time  to  “feel  out”  the  operational 
situation  and  make  adjustments.  The  authors  had  not  participated  in  the  analytical  first  steps  of  the  ISD 
curriculum  development.  The  ISD  team  had  expected  that  the  authors  would  be  subject  matter  experts  and 
would  be  aware  of  the  principles  of  programmed  instruction.  The  authors  expected  that  the  initial  versions 
of  criterion  tests,  teaching  points,  and  MVE  would  be  adequate.  The  authors  were  accustomed  to 
laissez-faire  management,  while  on  the  other  hand,  the  experience  of  the  ISD  team  had  shown  that  initial 
authoritarian  management  followed  by  a gradual  reduction  of  controls  as  course  personnel  accommodated 
to  new  concepts  and  procedures  was  more  effective.  With  such  differing  expectations  many  difficulties 
arose. 

Communication  problems  were  apparent  between  the  two  groups  after  ISD  assumed  lesson 
development  responsibilities.  For  example,  three  months  after  the  system  tryout  and  validation  phase  had 
begun,  some  authors,  including  those  responsible  for  collecting  feedback  information,  could  not  describe 
the  procedures  for  validating  a lesson.  Apparently,  neither  element  within  the  ISD/PLATO  team  was 
completely  successful  at  coordinating  evaluation  activities. 

The  primary  mission  of  a technical  training  center  is  the  training  of  students.  The  intervention  of  an 
experimental  medium  or  innovative  program  into  such  an  environment  may  produce  stress  because  of  its 
apparent  interference  with  the  primary  mission.  Consequently,  the  new  program  most  often  receives  a low 
priority  when  action  is  required.  Tire  impact  of  “resistance  to  change”  on  the  instructional  materials 
development  process  may  be  noted  in  subsequent  paragraphs  of  this  section,  as  well  as  the  courseware 
development  and  authoring  sections. 

The  major  goals  of  the  ISD  team  were  to  insure  training  relevancy  and  adequacy  while  striving  for 
maximum  training  efficiency.  Accomplishing  these  required  changes  within  organizations  which  were 
resistant  to  new  concepts.  The  intervention  of  the  PLATO  research  program  in  this  situation,  with  its  many 
unique  needs  and  problems,  acted  as  a multiplicative  factor.  School  action  on  project-related  needs  was 
correspondingly  slower.  For  example,  two  author  positions  remained  vacant  for  over  9 months.  Requests  to 
remove  several  project  personnel  from  additional  duties  were  not  acted  upon  in  a timely  manner.  Project 
personnel  action  to  ameliorate  the  problem  involved  external  pressure  and  internal  salesmanship.  Sustained 
interest  in  the  project  from  ATC  Headquarters  and  internal  promotion  of  the  project  within  the  technical 
school  hierarchy  gradually  increased  organizational  support. 

The  management  of  lesson  development  lacked  specific  direction.  Aside  from  a general  experiential 
learning  goal,  the  project  was  not  committed  to  a specific  goal.  Project  direction  vacillated  between  research 
and  operations,  depending  on  the  agency  providing  directional  assistance.  Fortunately,  this  conflict  was 
recognized  and  decisions  concerning  the  issue  were  brought  out  after  the  May  1974  project  review. 

The  operational-research  conflict  had  its  repercussions  upon  the  ISD  efforts,  where  adaptation  of 
educational  innovations  was  a strength  of  the  systems  approach.  Within  this  PLATO/ISD  project,  the 
emphasis  on  operational  concerns  took  priority  over  innovative  uses  of  the  computer,  resulting  in 
conservative  lesson  development. 

Some  authors  were  distraught  over  the  sterility  of  the  PLATO  lessons  prepared  within  the  ISD 
managerial  scheme.  ISD  personnel  believed  PLATO  to  be  like  most  other  media  at  their  disposal; 
consequently.  PLATO  lessons  were  developed  using  similar  techniques.  On  the  other  side,  some  PLATO 
authors  felt  that  ISD  people  were  not  knowledgeable  in  the  area  of  CAI  lesson  development  and  objected  to 
being  restricted  to  a particular  style  and  set  of  values.  Being  linear  and  lean,  the  lessons  did  not  tax  the 
capabilities  of  the  PLATO  system.  The  lessons  were  to  be  augmented  with  branching  based  on  student 
response  and  history,  but,  in  most  cases,  these  strategies  were  not  necessary  to  achieve  validation. 

Hardware-  and  Software-Related  Constraints.  The  TUTOR  language  used  to  code  the  PLATO  lessons 
was  adequate  for  the  needs  of  the  site.  This  observation  was  predicted  upon  the  conditions  of  lesson 
strategics  and  site  personnel.  In  general,  the  more  innovative  a lesson,  the  greater  the  demand  on  an 
author/codcr's  knowledge  of  the  language.  Most  lesson  authors  had  little  trouble  coding;  therefore,  they 
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required  assistance  with  only  the  most  difficult  portions.  The  data  collection  and  analysis,  the  on-line  block 
testing  and  analysis,  as  well  as  some  student  management  and  administrative  routines,  were  effectively 
handled  by  the  coders.  If  lessons  and/or  routines  were  more  complex  and  coders  were  not  available,  the  site 
was  hard  pressed  to  prepare  them  efficiently. 

Hardware  limitations  hindered  the  development  of  instructional  materials.  The  delayed  installation  of 
a microwave  link  limited  the  number  of  operable  terminals  to  nine  until  October  1974.  File  space  on  disk 
memory  and  shortages  of  ECS  delayed  coding.  Design  deficiencies  in  the  slide  selector  made  frequent 
adjustments  necessary  and  further  slowed  the  validation  process.  High  user  demands  on  the  central 
processing  unit  and  ECS  increased  response  latency  and  lesson  condensing  times,  which  were  very  irritating 
to  authors.  These  cyclical  periods  of  hardware  resource,  abundance  and  famine,  reliability  and  unreliability, 
characterized  the  evolution  of  the  PLATO  system. 

Courseware  Development.  Throughout  the  entire  project,  microfiche  procurement  was  a perplexing 
constraint.  It  essentially  took  one  full-time  instructional  programmer  several  months  to  refine  the  process 
until  the  final  product  could  be  attained  within  a suitable  time  frame.  Even  then,  it  required  in-person 
coordination  at  CERL,  including  personal  handling  to  limit  mailing  delays. 

Quality  control  of  the  final  microfiche  products  should  permit  only  microfiche  suitable  for  long-term 
student  use.  The  test  microfiche  used  during  validation  had  resolution  and  color  distortion  problems.  Some 
slide  images  were  retaken  because  of  poor  contrast,  unreadable  text,  and  other  difficulties. 

The  lack  of  subject  matter  expertise  within  the  author  group  necessitated  several  adjustments, 
including  the  necessity  for  authors  to  review  subject  matter  thoroughly.  Additional  research  impeded  the 
rate  of  individual  lesson  development,  thus  reducing  the  scope  and  delaying  the  tryouts  of  the  automated 
management  and  administrative  routines  programs. 

The  lesson  development  process  had  new  lessons  placed  “on  the  shelf’  after  initial  student  tryouts 
until  the  validation  and  tryout  phase  began.  Many  of  the  authors  disliked  the  fact  that  their  lessons  were 
not  being  used  on  a continuing  basis  by  the  course.  They  would  have  preferred  to  receive  continuing 
student  feedback  on  their  lessons. 

A complex  issue  in  the  courseware  development  was  the  depth  of  training  required  for  an  apprentice 
mechanic.  This  issue  is  not  specific  to  a PLATO-based  training  system  and  is  certainly  not  unusual  in  ISD 
efforts.  However,  it  was  a recurring  theme  in  the  project  personnel  interviews  and  of  importance  to  course 
personnel.  Information  for  determining  training  scope  and  depth  came  from  a variety  of  sources  such  as  the 
occupational  survey  report  (OSR),  field  evaluations,  instructor  field  experience,  Specialty  Training 
Standard  (STS),  et  cetera.  The  scope  of  training  depended  primarily  on  a quantitative  basis  (OSR  and  field 
evaluation),  while  the  basis  for  depth  of  training  was  qualitative.  That  is,  quantitative  data  establishing  the 
percentage  of  technicians  performing  a task  were  obtainable:  however,  the  level  at  which  they  should 
perform  after  graduating  from  technical  school  was  the  consensus  of  opinion  and  often  arbitrarily 
interpreted.  The  depth  of  training  may  not  be  set  appropriately.  Two  safeguards,  formative  product 
evaluation  and  field  evaluation,  are  built  in  the  ISD  model  to  insure  adequacy  of  training.  In  developing 
instruction,  formative  product  evaluation  enhances  the  probability  that  background  and  specific 
task -related  cognitive  information  attains  the  minimum  level  that  allows  a student  to  achieve  the  standard 
of  the  objective  associated  with  the  task.  Field  evaluation  insures  that  the  standard  of  the  objective  is 
representative  of  the  predesignated  level,  the  STS  “2b  level"  for  example.  Herein  lies  the  problem  that  may 
negate  these  safeguards.  Real-world  constraints  contaminate  the  interpretation  of  the  performance  level. 
For  example,  it  may  not  be  practical  to  have  a student  perform  all  parts  of  a task;  i.e.,  he  may  only  do  a 
third  of  it  in  a three-man  group,  or  even  less  if  he  is  a follower  and  allows  others  to  do  his  portion.  After  the 
task  is  performed,  the  group  receives  a satisfactory  rating  and  is  allowed  to  progress.  The  STS  standard  for 
most  tasks  is  “2b,"  which  means:  “Can  do  most  parts  of  the  task,  needs  help  only  on  the  hardest  parts.  May 
not  meet  local  demands  for  speed  or  accuracy.”  This  is  a terminal  level,  the  entry  behavior  of  the  student 
into  the  OJT  program.  None  of  the  students  in  this  group  truly  demonstrated  that  they  met  the  standard. 
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They  probably  would  be  able  to,  but  there  is  no  tangible  evidence  that  they  can;  thus,  in  some  cases,  they 
may  not.  In  those  cases,  the  training  prior  to  the  criterion  test  is  suspect,  however,  the  formative  evaluation 
will  not  detect  this  for  the  students  have  technically  achieved  criterion.  One  impractical  alternative  is  to 
impose  a requirement  that  each  student  will  actually  perform  most  parts  of  a task.  The  ramifications  of 
such  an  action  in  relation  to  increased  training  time  and  equipment  make  the  option  prohibitive.  Moreover, 
if  the  training  is  really  inadequate,  the  field  evaluation  should  detect  inadequacies. 

Field  evaluations  are  highly  reactive  measures;  that  is,  the  dependent  variable  is  influenced  by  the 
measurement  device.  A shop  chief  who  is  filling  out  a field  survey  and  must  recall  information  about  an 
airman  who  has  been  with  him  for  several  months  relies  on  more  factors  than  the  airman’s  ability.  How  his 
observation  is  influenced  by  his  assessment  of  the  man’s  character,  isolated  instances,  overall  ability, 
unwillingness  to  make  negative  ratings,  etc.,  is  uncertain.  At  best,  the  field  evaluation  is  a gross  assessment. 
Only  if  training  is  dramatically  deficient  will  the  evaluation  detect  it. 

Much  current  training  may  border  in  the  grey  zone  between  the  minimal  training  philosophy  and 
training  inadequacy.  Course  personnel  and  project  authors  frequently  expressed  concern  about  insufficient 
depth  of  training  during  the  pre-extension  period.  The  ISD  team  accommodated  somewhat,  but  adhered  to 
the  minimal  training  philosophy  since  the  evolutionary  facet  of  the  ISD  model  will  remove  discrepancies. 

Authoring.  As  the  lesson  development  model  description  depicted,  the  author  retained  the  primary 
role  in  the  preparation  of  a CA1  lesson.  From  the  author’s  viewpoint,  all  higher  levels  of  the  organization 
existed  to  assist  in  the  performance  of  author  duties  and  to  insure  that  courseware  development  was 
expeditiously  accomplished.  It  was  at  this  level  where  all  lesson  development  constraints  and  issues  were 
most  keenly  felt  and  many  authors  were  openly  critical  of  their  working  environment.  One  aspect  of  their 
environment  became  a focal  point  for  attention.  Dissatisfaction  with  work  incentives  was  necessarily 
expressed  from  the  time  the  project  was  a year  old.  For  instance,  interest  in  obtaining  an  adequate  job 
description  intensified  during  the  transition  to  ISD  management.  Civilian  authors  were  very  concerned 
about  their  promotion  future  and  relative  standing  with  instructors  who  actually  were  performing  in-class 
instructional  duties.  They  desired  documented  credit  for  their  extensive  efforts  under  difficult 
circumstances  and  for  their  acquired  expertise.  However,  technical  school  management  had  no  real 
justification  or  need  for  creating  a job  description  depicting  the  unique  duties  of  a PLATO  author. 
Moreover,  one  intention  of  the  service  test  was  to  examine  the  feasibility  of  using  lay  instructors  as  authors, 
thus  increasing  the  potential  for  cost  effectiveness  by  reducing  production  costs. 

Intrinsic  work  incentives  remained  consistent  throughout  the  service  test.  CAI  authoring  allowed  the 
individual  to  be  creative,  avoiding  the  somewhat  repetitious  activities  of  conventional  instructing.  Most 
authors  felt  the  problem-solving  nature  integral  to  CAI  lesson  development  posed  challenges  which  made 
the  work  more  interesting.  Nevertheless,  these  intrinsic  motivating  aspects  of  authoring  lessons  diminished 
as  task  familiarity  and  frustrations  accumulated. 

As  the  validation  of  new  courseware  neared  completion,  so  too  did  the  initial  training  of  Chanute  CAI 
lesson  authors.  Though  most  had  worked  on  lesson  preparation  for  several  years,  a finalized  product  was 
never  achieved,  due  to  the  changeable  service  test  conditions.  The  authors  had  received  most  of  their 
training  through  the  trial  and  error  method  and  an  assortment  of  ad  hoc , self-study  programs.  Even  after 
the  merger  of  the  PLATO  and  ISD  projects,  author  training  was  inadequate,  for  the  ISD  team  had 
overestimated  the  available  expertise.  As  late  as  the  validation  stage,  the  need  for  training  was  apparent. 

Lesson  Characteristics.  A detailed  report  was  prepared  by  CERLat  the  request  of  the  PLATO  project 
evaluation  team  on  lesson  characteristics  (MTC  Report  10,  1977).  The  interested  reader  will  find  these 
reports  quite  helpful  in  making  lesson  design,  preparation,  and  evaluation  decisions.  During  lesson 
validation,  the  project  evaluators  felt  that  a detailed  review  of  a sample  of  the  Chanute  lessons  before  and 
after  validation  would  reveal  important  insights  to  CAI  courseware  design.  Moreover,  this  would  satisfy  an 
ancillary  goal  of  obtaining  a detailed  description  of  the  Chanute  lessons  for  evaluative  purposes.  This 
section  is  not  intended  to  overlap  the  findings  contained  within  the  CERL  reports,  but  to  elaborate  and, 
where  important,  reinforce  their  general  findings. 
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This  section  covers  four  major  CAI  courseware  areas.  Instructional  Strategies,  refers  to  the  basic 
format  of  the  lessons  (tutorial,  drill  and  practice,  simulation,  etc).  Interaction  refers  to  the  exchange  of 
information  between  student  and  lesson.  Instructional  Programming  Features  discusses  presentation 
techniques  formating  of  displays,  student  affective  consideration  and  the  like.  Finally,  student  evaluation 
pertains  solely  to  the  end-of-lesson  testing  procedure. 

In  no  way  was  the  effectiveness  of  any  lesson  feature  systematically  investigated.  Comments  were 
collected  from  a sample  student  critique  form,  but  no  frequency  data  were  obtained.  Evaluative  statements 
should  therefore  be  regarded  as  considerations  and  should  not  be  interpreted  as  hard  guidelines. 

Instructional  Strategies.  Virtually  every  lesson  developed  for  mainstream  instruction  followed  the 
simple  tutorial  learning  activity  model.  Levien  (1972,  p.  355)  described  a simple  tutorial  as  being  composed 
of  “a  linear  series  of  factual  statements  interspersed  with  predetermined  questions  and  responses.  Each 
student,  regardless  of  ability,  performance,  or  prior  knowledge  is  required  to  proceed  through  the  same 
materia.”  Typically,  the  simple  tutorial  closely  resembled  a programmed  i , used  few  CAI  system 
capabilities,  and  was  often  denigrated  as  being  computer-assisted  page  turning.  This  model  was  selected  for 
a number  of  reasons,  the  first  being  expediency.  From  a writing  and  computer  coding  standpoint,  this  was 
the  easiest  type  of  lesson  to  prepare.  Mean  initial  development  time  dropped  by  a factor  of  approximately 
one-half  to  100  hours  per  student  contact  hour.  Second,  the  level  of  subject  matter  complexity  was  low,  as 
was  the  requirement  for  comprehension  and  retention.  For  the  tasks  that  graduates  performed  in  the  field 
at  the  apprentice  level,  a strong  cognitive  base  was  not  required.  Hence  th-'  tutorial  paradigm  permitted 
most  vehicle  maintenance  students  to  achieve  the  desired  level  of  comprehension.  Higher  order  lesson 
strategies  were  not  needed  for  most  subject  matter.  CAI  lessons  met  the  same  90  percent  validation 
criterion  as  programmed  texts.  Since  augmentation  of  a lesson  is  generally  needed  for  lower  ability 
students,  higher  ability  students  are  forced  to  wade  through  amplifications  of  the  material.  This  suggests 
that  a more  complex  tutorial  should  be  used  in  a totally  self-paced  system.  PLATO  lessons  with  tracks  for 
students  of  varying  ability  were  successfully  employed  in  an  Army  technical  training  environment  (Dept  of 
the  Army,  1975). 

Some  mainstream  lessons  used  drill  and  practice  strategies.  One  lesson  required  the  student  to 
identify  each  battery  ignition  system  component  before  proceeding  to  subsequent  lesson  modules.  The 
sequence  of  microfiche  image  projection  was  random  to  avoid  a serial  position  learning  effect. 

The  student  was  able  to  exercise  some  learner  control  over  his  path  through  about  half  the  lessons  by 
a lesson  index  that  allowed  the  student  to  choose  the  order.  Even  though  this  feature  was  present,  most 
students  followed  the  prescribed  sequence.  The  index  was  valuable  in  that  it  allowed  the  student  to 
selectively  review  portions  of  a lesson. 

Almost  all  lessons  contained  a provision  to  allow  a student  to  page  backward  through  a lesson.  This 
was  not  an  original  design  feature,  though  the  need  quickly  materialized.  Students  frequently  wanted  to 
review  previously  displayed  material. 

One  learner  control  feature  which  was  removed  was  the  lesson  by-pass  test.  Many  students  would 
select  this  option  even  if  they  had  limited  entering  knowledge  concerning  the  subject  area.  They  did  this  to 
familiarize  themselves  with  the  end-of-lesson  test,  keying  in  on  the  test  items.  Since  most  tests  sampled  the 
content  area,  it  became  necessary  to  delete  this  option. 

Few  adaptive  strategies  were  built  into  the  lessons.  Branching  consisted  mainly  of  forced  review.  The 
student  was  required  to  repeat  a segment  of  a lesson  if  response(s)  to  a sub-terminal  exercise  were  not  to 
the  standard  of  the  author.  For  most  lessons,  if  the  student  did  not  achieve  the  end-of-lesson  test  standard, 
he  was  required  to  reaccomplish  the  entire  lesson.  Some  lesson  tests  were  programmed  to  determine  which 
lesson  modules  the  students  should  review  and  then  automatically  routed  them  through  an  individualized 
review. 

Lessons  which  employed  more  of  the  flexibility  offered  by  PLATO  were  prepared  during  the  first 
project  phase.  A part-task  simulation  of  a laboratory  exercise  required  the  student  to  practice  battery 
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hydrometer  reading  skills.  A dialogue  strategy  was  chosen  to  reinforce  starter  system  diagnostic  skills.  These 
lessons  demonstrated  that  these  strategies  could  be  developed  by  an  experienced  technical  training  author 
or  author  team.  The  gencralizability  is  somewhat  limited  though  owing  to  the  background  of  the  authors 
responsible:  college  education,  interest  in  education,  and  intrinsic  motivation.  The  motivation  is  important, 
because  such  lessons  are  very  difficult  to  design,  code,  and  evaluate. 

More  creative  lesson  strategies  were  not  used  principally  because  of  Service  Test  conditions.  It  can  not 
be  determined  if  other  strategies  could  have  produced  greater  instructional  efficiency  gains:  however  the 
simple  tutorial  strategy,  when  used  to  present  facts  and  principles,  was  effective,  and  could  be  efficiently 
prepared. 

Interaction.  The  continuous  exchange  of  information  and  responses  between  student  and  medium  is 
one  of  the  major  purported  advantages  of  CA1.  From  observation  of  students  in  the  PLATO  laboratory,  it 
seemed  that  interaction  was  motivational  and  promoted  learning  for  the  majority  of  vehicle  maintenance 
students.  For  the  purposes  of  this  report,  interaction  will  include  those  lesson  techniques  which  elicit 
student  responses,  PLATO  system  response  judging,  and  author-generated  feedback. 

The  mainstream  lessons  used  three  major  interaction  modes,  multipl-choice.  fill  in  the  blank,  and 
true/false  questions.  Matching  routines  were  employed  less  frequently,  used  most  often  in  end-of-lesson 
tests,  and  were  more  difficult  to  incorporate  into  the  lesson  coding  even  though  a special  routine  could  be 
adopted. 

The  frequency  and  distribution  of  interaction  modes  varied  considerably.  Table  31  (adapted  from 
Klccka,  1977)  depicts  the  interaction  modes  and  the  ratio  of  interaction  to  individual  lesson  displays.  For 
this  sample  of  eight  lessons,  the  ratio  of  interaction  to  textual  display  is  about  one  to  three;  i.e.,  on  the 
average  the  student  examined  three  instructional  displays  and  then  received  a question.  Several  of  these 
lessons  used  massed  practice  questions  which  were  clustered  at  the  end  of  a lesson  section.  Research  in  the 
area  of  distribution  of  interaction  generally  has  shown  that  for  most  subject  matter  types,  levels  of  subject 
matter  complexity,  student  abilities,  etc.,  spreading  the  interaction  is  preferable.  Moreover,  it  is  desirable  to 
have  the  student  relate  to  all  new  information.  Since  many  validated  Chanute  lessons  violated  these  rules  of 
thumb,  it  was  decided  that  the  statistical  relationship  between  degree  of  interaction  and  longitudinal  lesson 
effectiveness  should  be  examined.  Percentage  of  interaction  was  correlated  with  student  failure  rates,  r(6)  = 
.49.  p > .05.  The  high  probability  of  chance  occurrence  limits  further  discussion  to  speculation.  If  in  fact  a 
moderate  positive  relationship  exists  between  these  lesson  attributes;  i.e..  high  interaction  rates  are 
associated  with  high  failure  rates,  it  is  probably  because  of  their  relationship  to  at  least  one  concomitant 
variable:  subject  matter  complexity,  in  particular.  Any  future  research  design  which  sought  to  study  the 
frequencies  and  disposition  of  interaction  with  these  lessons  must  contain  appropriate  controls. 


Table  31.  Frequency  of  Lesson  Interaction  Types  and 
Interactive  Display  Percentage  Rates 


Lesson  Title 

Fill 

In 

Interaction  Typa 

Multiple 

Choice 

True/ 

False 

% Interactive 
Displays 

Emission  System 

2 

12 

i 

36 

Starter  System 

13 

33 

0 

48 

Transmissions 

0 

14 

n 

58 

Diesel  Engines 

8 

6 

0 

19 

Principles  of  Hydraulics 

3 

20 

5 

38 

Drive  Shafts 

3 

8 

1 

38 

Power  Takeoffs 

1 

2 

4 

43 

Electrical  Fundamentals 

1 

7 

0 

33 

64 
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The  TUTOR  language  had  an  impressive  response-judging  capability;  however,  it  was  not  extensively 
utilized  by  the  lesson  authors  during  initial  lesson  development.  Ideally,  data  gathered  during  lesson 
validation  would  be  used  to  modify  response-judging  flexibility  of  an  item.  Unfortunately,  lesson  revision 
all  too  often  consisted  of  quick  fixes.  All  too  often  the  multiple-choice  judging  routine  would  not  be 
programmed  to  provide  appropriate  feedback  or  route  the  student  to  review  after  several  incorrect 
combinations  of  answers.  Similarly,  a fill-in-the-blank  item  judger  would  often  not  accept  synonyms  or 
close  misspellings.  With  inadequate  feedback  and  no  bypass  option,  a deadend  would  result:  the  student 
could  not  proceed,  go  back,  or  ask  for  assistance. 

Students  often  commented  on  fill-in-the-blank  items.  Along  with  the  response-judging  inflexibility, 
they  disliked  typing  responses  because  of  keyboard  unfamiliarity  and  because  they  were  required  to  recall 
and  spell  newly  introduced  terminology.  For  these  reasons  and  because  fill-in  questions  were  more 
time-consuming  to  code,  they  were  not  often  used.  Also  multiple-choice  questions  are  typically  selected  for 
block  tests  and  it  was  felt  that  the  lesson  and  test  styles  should  be  similar. 

The  feedback  given  to  a student  varied  widely  as  a function  of  the  desires  of  the  author,  experience, 
and  work  conditions.  If  under  pressure  to  complete  a lesson,  for  example,  the  author  might  rely  solely  on  a 
very  elementary  form  of  feedback:  a “yes,”  or  “no,”  or  “ok”  after  the  student  initiated  judging.  Under 
other  circumstances,  the  author  provided  feedback  which  was  tailored  for  each  possible  item  response  as  in 
multiple-choice  questions.  As  a group,  the  lessons  did  not  employ  good  feedback  techniques.  The 
instructional  efficiency  of  these  validated  lessons  could  have  been  substantially  increased  if  general 
instructional  programming  guidelines  were  followed. 

PLATO  offers  a rich  potential  for  interaction  between  student  and  courseware.  The  Chanute  lessons 
did  not  tap  this  potential  extensively;  consequently,  there  was  room  for  improvement  within  each  facet  of 
interaction.  It  was  believed  that  the  efficiency  of  the  entire  lesson  population  could  have  been  substantially 
increased  if  deficiencies  were  corrected  and  generally  accepted  technqiues  followed  during  modification 
efforts.  This  could  have  been  done  systematically  so  that  research  design  related  to  instructional 
programming  techniques  might  have  been  performed. 

Instructional  Programming  Features.  The  PLATO  system  provides  graphics,  microfiche  projection, 
touch  panel,  and  audio  capabilities  for  use  in  lesson  delivery.  With  the  exception  of  the  latter  two 
capabilities,  which  were  unavailable  until  extremely  late  in  the  project,  these  presentation  techniques  were 
frequently  utilized  in  the  lessons  developed  for  the  Special  Purpose  Vehicle  Repairman  courses.  The  various 
presentation  techniques  were  selectively  used  to  best  exploit  their  perceptual,  instructional  programming, 
affective,  and  procedural  capabilities. 

Within  the  perceptual  area,  the  graphics  and  microfiche  capability  were  used  to  reduce  the  need  for 
training  aids,  especially  actual  pieces  of  operational  equipment  adapted  for  training  use.  The  graphical 
representation  of  equipment  ranged  substantially  as  to  realism,  but  the  effect  of  this  variable  on  transfer  to 
a task  such  as  component  identification  was  not  investigated.  Students  frequently  commented  that  they 
liked  the  graphics,  especially  when  ccompanied  by  animations.  It  was  felt  that  graphics  animation  had  a 
definite  advantage  for  teach;  g certain  system  operation  concepts  and  movable  component 
interrelationships. 

Microfiche  application  was  somewhat  limited  by  its  poor  resolution  and  registration  (image  position) 
problems.  In  a typical  lesson,  the  slide  would  appear  along  with  textual  material  written  in  a darkened 
portion  of  the  plasma  screen.  Gross  slide  features  were  typically  discernible,  but  fine  details  sometimes  were 
not.  even  when  the  student  manipulated  the  focus  wheel.  Overwriting  the  image  was  a problem,  since  bright 
lights  washed  out  the  contrast  between  the  orange  writing  and  the  image.  Moreover,  the  position  of  the 
slide  image  varied  considerably. 

For  parts  identification,  microfiche  should  be  used,  while  graphics  are  generally  better  adapted  to 
instructional  goals  that  do  not  require  visual  realism,  or  when  additional  realism  is  distracting.  Many  authors 
expressed  that  they  would  have  liked  to  use  microfiche  in  this  way,  but  found  that  the  preparation  process 
was  too  lengthy. 
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One  observation  related  to  the  presentation  of  textual  material  is  worth  emphasizing.  Displays  which 
plot  slowly  are  distracting.  Unless  the  technioue  which  causes  a reduced  plotting  speed  has  a clear 
instructional  value,  it  should  not  be  employed. 

Although  several  copies  of  Educational  Psychology  by  Anderson  and  Faust  (1973),  were  available  to 
the  authors,  few  of  their  instructional  concepts  were  noted  in  the  lessons.  In  part,  this  condition  existed 
because  of  the  high  attrition  rate  among  the  more  experienced  authors.  These  authors  had  reached  a plateau 
in  their  coding  proficiency  and  were  ready  to  learn  more  of  the  instructional  programming  concepts. 
Another  reason  was  the  necessity  to  complete  lesson  development  activities  before  the  first  tryout  class. 
One  abused  concept  was  that  of  avoiding  the  copy  frame.  Unless  the  instruction!1  goal  was  to  teach  new 
terminology,  this  programming  technique  was  of  limited  value.  Very  few  questions  required  the  student  to 
think  about  the  material;  most  required  him  to  parrot  it.  This  activity  may  explain  a student  criticism  of 
some  end-of-lesson  test  items.  Numerous  complaints  were  made  that  the  test  questions  did  not  say  things  in 
the  same  way  as  they  were  stated  in  the  lesson.  It  seemed  that  these  students  were  not  acquiring  concepts 
for  there  was  no  reason  to  suspect  that  many  test  item  stems  were  ambiguous.  They  were  associating  words. 

Though  the  author  group  had  limited  expertise  in  tire  use  of  instructional  programming  concepts, 
they  were  more  knowledgeable  than  the  majority  of  1SD  team  SMEs  who  generate  materials  for  ISD 
projects.  These  materials  typically  demonstrate  the  same  lack  of  expertise.  It  is  unfortunate  that  the  IPs 
who  monitored  the  work  of  these  SMEs  did  not  provide  sufficient  training  and  allowed  the  courseware 
produced  to  contain  violations  of  programming  concepts.  It  was  apparent  that  some  PLATO  project  IPs  had 
a limited  knowledge  in  this  facet  of  their  occupation.  This  belief  will  be  reinforced  later  when  validation 
findings  are  discussed. 

One  criticism  of  CAI  and  programmed  instruction  in  general  was  that  it  was  cold  and  impersonal.  In 
hopes  of  producing  good  affect,  many  of  the  authors  interjected  humor  and  personalization  into  their 
lessons.  Personalization  typically  refers  to  the  technqiue  whereby  the  student’s  name  is  displayed  within  the 
text  he  is  reading.  Since  students  in  the  PB  and  CP  condition  did  not  perceive  PLATO  training  as 
mechanistic,  these  techniques  may  be  considered  as  partially  successful. 

Several  procedural  aspects  were  related  to  materials  presentation.  Students  were  allowed  to  take  notes 
if  they  desired,  even  though  this  lengthened  lesson  completion  times.  Early  in  the  initial  tryout,  students 
stated  that  they  were  anxious,  because  they  had  nothing  to  review  for  the  block  tests.  To  alleviate  this 
anxiety,  note  taking  was  allowed. 

Student  Evaluation.  End-of-lesson  tests  performed  at  least  three  functions.  They  established 
minimum  standards  for  student  achievement,  provided  information  for  lesson  improvement,  and  aided  in 
diagnosing  individual  student  difficulties.  Tire  format  of  most  end-of-lesson  tests  was  me’  hing,  with  some 
multiple-choice  items.  Students  were  allowed  to  change  any  answer  before  they  initiated  judging.  If  the 
student  passed  the  test  (criterion  was  typically  80  percent  of  the  items),  they  proceeded  to  the  next 
learning  module.  Upon  failure,  they  were  required  to  repeat  all  or  part  of  a lesson. 

Some  students  complained  because  they  were  not  given  feedback  if  they  passed  the  test.  Feedback 
was  not  provided  for  three  reasons.  First,  the  test  sampled  the  instructional  content  area;  therefore,  it  was 
not  beneficial  to  limit  student  attention  to  a portion  of  the  lesson.  Second,  students  received  the  same  test 
version  each  time;  therefore,  they  could  remember  the  correct  answer  without  comprehension.  Third, 
students  would  be  more  prone  to  compromise  the  test  if  they  were  sure  of  the  correct  answers.  This 
procedure  for  assessing  student  performance  worked  reasonably  well.  Most  students  were  not  aware  of  the 
one  or  two  items  that  they  missed,  nor  why  they  were  missed.  The  content  and  structure  of  the 
end-of-lesson  test  indicated  the  low  learning  level  required  of  the  student.  The  field  evaluation  depicted  that 
the  four  courses  were  successfully  performing  their  role;  thus,  it  can  be  surmised  that  substantial  task 
background  knowledge  was  not  required  to  perform  adequately  in  the  field. 

Lesson  Validation.  Ideally,  the  lesson  validation  process  involves  the  analysis  of  student  tryout  data 
to  make  improvements  to  a lesson  until  90  percent  of  the  students  sampled  attain  the  standard  specified  in 
the  lesson  objective.  Objective  attainment  is  assessed  by  the  MVE  which  later  becomes  the  end-of-lesson 
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test.  The  test  determines  the  desired  learning  level.  The  role  of  the  lesson  author  is  to  revise  instruction 
until  the  validation  criteria  are  met.  The  decision  is  made  after  consulting  with  or  observing  students,  and 
looking  at  the  student  responses  within  that  lesson  module.  Since  the  test  samples  a portion  of  the  content 
areas,  the  author’s  revision  technique  is  applied  to  the  entire  section,  not  just  the  sample  which  is  covered 
by  the  validation  examination.  Ideally,  the  outcome  of  the  validation  is  a refined  product  suitable  for  above 
90  percent  of  the  student  population. 

Validation  Techniques.  The  lesson  revisions  consisted  primarily  of  obtaining  passing  scores,  rather 
than  insuring  comprehension  and  promoting  retention  of  material.  When  a lesson  was  long  — usually  greater 
then  1 hour  — it  was  divided  into  as  many  as  four  parts,  each  with  its  own  separate  test.  This  served  three 
purposes:  (a)  to  promote  retention  by  limiting  interference  of  new  and  old  related  materials  (proactive  and 
retroactive  inhibition);  (b)  to  decrease  the  time  between  presentation  and  measurement,  minimizing 
forgetting;  and  (c)  to  increase  attention  since  practical  experience  indicated  that  students’  attention 
diminished  if  they  were  made  to  interact  with  the  same  material  for  long  periods.  Of  the  33  lessons,  only 
one  exceeded  an  average  completion  time  of  1 hour. 

Another  technique  to  increase  the  frequency  of  passing  test  scores  involved  overprompting  the 
students  (Remember  this  for  the  test!).  Sometimes  the  prompting  would  be  more  subtle,  but  still 
identifiable  to  the  student.  To  illustrate,  statements  related  to  test  items  were  italicized  in  some  lessons. 

The  most  effective  test-teaching  approach  was  keying  interaction  items  directly  to  the  test  items 
which  were  missed.  Thus,  a multiple-choice  question  might  be  inserted  to  provide  practice  for  a matching 
test  item. 

The  only  major  change  to  the  organization  of  the  lessons  as  a result  of  the  validation  process  was  the 
introduction  of  indices.  These  allowed  the  students  to  select  their  own  path  through  the  lesson  and 
selectively  review  sections.  Minor  changes  consisted  of  correcting  grammatical,  spelling,  and  technical 
enors. 

No  general  guidelines  for  lesson  presentation  techniques  were  ascertained  from  the  study  of  lesson 
changes  made  to  achieve  validation  criteria.  Instead,  it  was  noted  that  the  true  intent  of 
validation-formative  lesson  evaluation -had  been  reduced  to  an  exercise  of  teaching  a test.  The  long-term 
effects  of  circumventing  the  role  of  validation  were  apparent  after  all  lessons  were  validated.  Many  lessons 
still  had  grammatical  and  technical  errors,  and  student  comments  related  to  poor  comprehension  were 
found  in  the  critique  forms. 

Conclusions 

The  following  conclusions  follow  from  the  observations  of  the  project  evaluation  team: 

1 . The  process  used  to  develop  PLATO  materials  during  the  second  phase  of  the  project  was  more 
effective  and  efficient  than  that  used  in  Phase  1. 

2.  The  lessons  prepared  did  not  fully  exploit  many  potential  PLATO  capabilities,  chiefly  because  of 
resource  constraints,  a lack  of  expertise  with  CAI,  and  inadequate  training  in  instructional  programming 
concepts  and  techniques. 

3.  Role  specialization  contributed  to  increasing  the  efficiency  of  lesson  preparation.  The  concept  of 
an  author/ instructor  is  too  unwieldy  for  a technical  training  environment;  with  too  many  roles  to  perform, 
the  efforts  of  an  author  become  diluted.  The  author/instructor  concept  may  linger  through  efforts  to 
concentrate  training  and  to  provide  author  aids;  however,  the  evolution  should  be  toward  a team  specialist 
approach. 

4.  The  PLATO  system  was  very  flexible  in  terms  of  instructional  design  capabilities  and  lesson 
tryout  data  handling. 

5.  Sophisticated  CAI  capabilities  may  not  be  necessary  for  efficient  lessons  which  present  low  order 
task  knowledges.  Low  order  task  knowledge  refers  to  the  condition  where  course  graduates  require  only  a 
limited  understanding  of  task-related  knowledge  to  perform  field  tasks  to  supervisor  satisfaction. 


■ 


67 


VII.  MANAGEMENT  OF  Till:  PLATO-BASED  COURSE 


This  section  will  attempt  to  pinpoint  the  extent  to  which  management  of  the  PLATO  resources 
contributed  to  project  effectiveness.  Management  outcomes  will  be  discussed  in  terms  of  daily  training 
activities,  student  control,  lesson  control,  instructor  roles,  and  test  management.  Comparative  evaluations, 
under  controlled  conditions,  of  various  management  schemes  were  beyond  the  scope  of  this  study. 
Consequently,  the  conclusions  share  the  limitations  of  the  methods  used  to  collect  the  data. 

Method 

Information  for  the  evaluation  of  management  practices  was  obtained  by  observational  and  interview 
techniques.  Informal  interviews  of  project  and  course  personnel  occurred  for  the  most  part  during  the 
summer  of  1976. 

Result  and  Discussion 

General  Management  Structure.  Lach  course  had  a course  :upcrvisor,  assisted  by  block  supervisors 
who  in  turn  were  assisted  by  instructor  supervisors.  All  were  qualified  instructors  and  could  assume 
classroom  duties.  Instructor  supervisors  normally  participated  as  instructors  in  addition  to  their  supervisory 
duties. 

In  the  area  of  personnel,  the  major  departure  of  the  PLATO  environment  from  standard  operating 
procedures  was  the  formation  of  a special  PLATO  author  group  which  consisted  of  three  people,  created  to 
maintain  the  PLATO  software.  Their  duties  primarily  centered  around  maintaining  lesson  materials  with 
corrections,  updates,  or  revisions  as  necessary,  keeping  the  student  router  functioning  to  collect  on-line 
data,  and  generally  acting  as  system  troubleshooters  to  keep  training  running  smoothly.  This  team  consisted 
of  two  authors  and  one  TUTOR  programmer. 

Maintenance  and  support  of  training  materials  would  normally  be  carried  out  by  the  Training  Services 
Division.  This  group  is  responsible  for  all  training  devices,  procedures  trainers,  textual  materials,  visual  aids, 
and  demonstration  models.  Under  normal  circumstances,  training  branches  have  no  responsibility  for  the 
maintenance  of  training  material  once  it  has  been  incorporated  as  a part  of  operational  training.  In  this 
regard,  establishment  of  a section  of  people  in  the  course  specifically  dedicated  to  maintenance  of  training 
material  was  a major  break  with  normal  management  procedures  in  resident  training.  For  a small  operation 
dealing  with  a very  limited  number  of  courses  within  a single  training  branch,  establishing  a dedicated  group 
of  people  was  probably  the  most  workable  management  scheme  possible 

This  management  scheme  for  general  maintenance  and  support  of  PLATO  training  was  judged  to  be 
effective  for  the  limited  application  observed.  If  wider  application  were  pursued,  a different  means  of 
supporting  the  software  would  be  necessary. 

Daily  Training  Management.  Student  attended  training  from  6 a.m.  to  noon  each  day.  Monday 
through  Friday.  Because  PLATO  was  down  for  maintenance  from  6 a.m.  to  7:40  a.m..  the  system  was  not 
available  for  the  full  training  shift.  Thus,  the  first  part  of  each  student’s  day  excluded  PLATO.  Instead,  the 
time  was  utili/.cd  with  off-line  instruction,  usually  in  the  laboratory,  but  frequently  in  class,  working  with 
special  tools  and  observing  demonstrations  conducted  with  models  and  cut-away  training  devices. 

Working  from  a lesson  plan,  which  will  be  described  more  fully  in  the  section  covering  management 
of  instructional  materials,  the  instructor  estimated  when  students  would  be  ready  to  proceed  to  the  next 
PLATO  session.  Usually  this  estimate  was  made  the  day  before  the  training  was  needed.  The  ins.ructor  then 
scheduled  the  class  by  making  an  entry  on  a large  scheduling  board  in  the  PLATO  lab  indicating  the  number 
of  students,  lessons  to  be  used,  and  approximate  start  and  finish  times.  There  were  occasions  when  schedules 
of  different  classes  conflicted.  These  conflicts  were  usually  resolved  by  compromise;  however,  an  informal 
priority  system  was  established  to  assist  in  determining  access  to  the  system.  First  priority  went  to  students 
ready  to  take  block  examinations  on  PLATO,  and  second  priority  went  to  classes  that  were  engaged  in 
operational  training;  third  priority  went  to  authoring  or  other  indirect  uses  of  the  system.  In  cases  where  a 
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conflict  existed  over  a few  terminals,  faster  students  who  were  working  with  enrichment  materials  were 
assigned  off-line  material,  while  the  rest  of  the  class  caught  up.  Tire  terminals  they  had  been  using  were  then 
taken  by  the  other  class  for  operational  training. 

Some  conflicts  arose  which  could  not  be  resolved  by  compromise.  These  problems  involved  the 
PLATO  system  itself  and  were  due  either  to  system  failure  or  to  insufficient  ECS.  When  the  system  failed 
and  it  appeared  that  it  would  be  inoperative  for  an  extended  period  of  time,  instructors  reverted  to 
stand-up  instruction  with  specially  prepared  back-up  lesson  outlines.  These  outlines  covered  the  same 
information  presented  in  the  PLATO  lesson,  but  it  was  delivered  via  traditional  lecture/discussion.  In 
addition  to  the  lecture  outline,  films  and  sound-slide  presentations  were  utilized  in  the  absence  of  PLATO. 

When  there  was  insufficient  ECS,  the  instructor  would  cither  rearrange  the  instructional  sequence 
slightly  to  postpone  the  PLATO  session,  or  revert  to  the  lecture  mode,  depending  on  time  or  sequence 
constraints. 

In  summary,  management  of  the  daily  training  activities  required  a great  deal  of  flexibility  from  all 
instructors.  Unexpected  system  failure,  shortages  of  ECS  or  terminals,  conflicts  of  schedules,  and  variations 
in  student  progress  through  the  curriculum  made  it  essential  to  leave  most  management  to  the  instructor. 

Under  circumstances  of  resources  limitations,  management  of  PLATO-bascd  training,  though 
imprecise,  was  perhaps  the  most  effective  possible.  Expanding  PLATO  to  other  operational  application 
would  entail  sizable  management  problems  that  would  have  to  be  approached  with  care  and  planning. 

Management  of  Student  Row.  Group-pacing  was  selected  for  the  service  test  application.  Pnat  is,  to 
optimize  the  time-saving  capabilities  of  PLATO,  students  in  a group  progressed  through  the  curriculum  at 
the  rate  of  the  slowest  group  member.  Previous  flow  management  used  the  lock-step  method  wherein 
students  were  allotted  a fixed  amount  of  time  for  each  major  objective  in  the  curriculum.  With  lock-step 
management,  instructors  were  sometimes  faced  with  a problem  of  having  too  much  time  for  a particularly 
fast  class  or  too  little  time  for  an  unusually  slow  group.  In  group-pacing,  the  instructor  could  progress  to 
the  next  objective  as  soon  as  he  felt  the  group  was  ready.  The  group-pacing  strategy  proved  more  flexible 
than  lock-step  and  generally  allowed  instructors  to  complete  training  in  significantly  less  time. 

On  the  other  hand,  one  of  the  major  advantages  of  lock-step  training  was  that  it  permitted  simpler 
management  controls.  Time  on  limited  equipment  could  be  scheduled  more  precisely,  there  were  fewer 
requirements  to  fall  back  on  alternate  contigencies.  and,  in  general,  it  was  much  simpler  for  supervisors  to 
manipulate.  Group-pacing,  on  the  other  hand,  placed  a greater  management  burden  on  the  individual 
instructor  and  left  much  of  the  control  of  training  in  his  hands. 

A third  alternative,  one  which  was  not  attempted  in  this  setting,  was  that  of  computer  self-pacing, 
where  individual  students  progress  through  the  course  at  their  own  individual  rate.  Several  problems 
precluded  self-pacing.  PLATO  was  limited  in  its  capability  to  support  simultaneous  use  of  several  different 
lessons.  The  number  of  lessons  allowed  in  ECS  (i.e.,  “condensed")  was  a function  of  the  number  of 
terminals  at  a PLATO  site.  Depending  upon  the  size  of  lessons  in  use,  only  around  ten  or  so  could  be 
condensed  at  any  one  time  and  still  be  within  ECS  limits.  The  net  effect  of  the  ECS  limitation  was  that 
self-pacing  might  not  have  been  a feasible  strategy,  since  it  would  not  always  have  been  possible  to  provide 
all  needed  lessons  at  a specific  time.  Another  difficulty  with  self-pacing  would  have  been  that  PLATO 
training  comprised  the  first  part  of  training,  and  the  latter  portion  was  designed  to  accept  students  on  a 
group  basis.  The  shred  areas  did  not  have  individualized  media;  consequently,  the  classroom  instruction  had 
to  be  conducted  by  instructors  in  a lecture  discussion  mode.  Moreover,  a heavy  investment  in  training 
equipment  was  required.  These  problems  prevented  self-paced  training,  though,  in  general,  self-pacing 
would  remain  the  most  efficient  means  of  getting  students  through  training. 

A group-paced  system  of  student  flow  has  some  distinct  disadvantages  along  with  the  advantages.  One 
of  these  disadvantages  is  a difficulty  posed  by  individual  differences  in  learning  speed.  There  could  be.  and 
often  were,  several  hours  difference  in  the  time  required  for  the  fastest  and  slowest  students  (in  a particular 
class)  to  finish  a group  of  lessons.  One  procedure  adopted  to  eliminate  student  waiting  time  was  to  have 
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those  who  finished  early  repeat  some,  or  all,  of  the  lessons.  This  served  to  keep  faster  students  occupied.  It 
also  insured  that  they  did  not  miss  some  important,  though  perhaps  untested,  aspects  of  the  lesson  material 
on  their  first  pass.  Unfortunately,  such  redundancy  may  have  had  a negative  impact  on  their  motivation 
since  there  was  no  reward  for  efficient  completion  of  materials.  Other  means  for  fdling  this  “waiting  time” 
were  to  allow  the  fast  students  to  review  “enrichment”  materials.  These  were  instructional  materials  which 
were  not  essential  to  achieving  the  required  (2b)  level  of  performance,  but  which  still  contributed  to  their 
general  competence  as  a mechanic. 

Sometimes  faster  students  were  placed  with  slower  students  to  act  as  “peer  instructors.”  Where  the 
faster  student  was  actually  a good  teacher,  this  was  an  effective  means  for  speeding  slower  students  along. 
On  the  other  hand,  many  times  the  student  peer  instructor  was  r.ot  so  much  interested  in  how  well  the 
“student”  learned  the  material  as  in  how  fast  the  student  completed  the  lesson.  As  a result,  the  slower 
student  sometimes  tended  to  rely  on  the  helper  too  much  for  the  answers  and  thereby  failed  to  learn  the 
material  as  well  as  if  left  on  his  own. 

Extremely  slow  students  were  brought  back  during  the  SIA  time  after  the  normal  training  day  to 
catch  up.  Such  a practice  was  not  encouraged,  however,  and  consequently  was  used  only  infrequently. 
Students  were  allowed  to  return  on  voluntary  SIA  to  make  up  time  lost  earlier.  As  a result  of  these 
procedures,  classes  usually  completed  the  PLATO  portion  of  the  course  at  approximately  the  same  time, 
within  2 to  4 days  of  each  other. 

In  summary,  management  of  student  flow  under  the  group-pacing  strategy  both  saved  time  and  was 
operationally  feasible.  It  posed  problems  for  coping  with  individual  differences  in  lesson  completion  rates, 
but  various  contingencies  were  developed  for  resolving  those  difficulties.  While  it  is  premature  to 
recommend  full-scale  self-pacing,  it  is  strongly  urged  that  the  strategy  be  implemented  on  a limited  basis  for 
at  least  one  course  to  make  a direct  determination  of  its  operational  feasibility. 

Management  of  Instructional  Material.  Control  of  student  learning  materials  was  done  principally  by 
the  instructor.  In  the  event  PLATO  was  u;. available  for  some  reason,  the  instructor  lectured  from  an  outline 
of  the  material  required  to  meet  the  training  objective.  If  PLATO  was  available,  the  instructor  would  assign 
his  students  to  a PLATO  module.  There  were  four  modules,  one  for  each  major  block  of  training. 

When  students  signed  onto  PLATO,  they  were  shown  a listing  of  all  the  lessons  in  their  current 
module.  They  had  to  complete  the  lessons  in  sequence  listed.  After  completing  each  lesson,  they  were 
returned  to  the  listing  display.  Students  could  not  go  to  a new  module  until  they  had  successfully 
completed  all  of  the  lessons.  If  they  completed  a module,  and  there  was  time  available,  the  option  was 
provided  to  return  to  a previous  module  and  review  other  lessons  completed  or  interact  with  enrichment 
material. 

Use  of  lesson  modules  enhanced  instructional  flexibility.  Instructors  generally  assigned  students  to  a 
subset  of  lessons  in  a module  at  each  session.  The  number  of  lessons  assigned  generally  depended  upon  the 
amount  of  time  available,  terminal  or  ECS  resources , and  daily  training  objectives.  It  was  necessary  for  the 
instructor  to  monitor  the  progress  of  the  students  to  be  sure  that  they  were  not  proceeding  too  fast  or 
spending  time  on  lessons  other  than  those  assigned. 

This  management  procedure  relied  rather  heavily  on  instructor  intervention.  Some  tasks  done  by  the 
instructor  could  have  been  assumed  by  PLATO  (i.e.,  monitoring  student  progress),  though  this  was 
probably  not  feasible  in  this  instance  because  of  the  terminal  and  ECS  limitations.  Fully  automating 
management  of  PLATO  material  might  also  have  produced  negative  side  effects.  For  example,  it  was  felt 
that  instructor  participation  in  the  training  process  was  an  important  element  in  maintaining  job 
satisfaction.  Under  the  intervention  mode  of  operation,  the  instructor  had  a substantial  and  active  role  in 
the  PLATO  training.  If  these  tasks  were  automated  and  not  directly  under  instructor  control,  there  would 
be  a loss  of  involvement.  The  potential  effects  of  this  loss  of  personal  involvement  were  reflected  in  the 
measures  of  instructor  attitudes  discussed  earlier. 
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In  spite  of  this  problem,  it  is  important  that  instructor  participation  in  routine  administrative  matters 
eventually  be  reduced  to  a minimum.  Instructor  involvement  is  important,  but  it  should  be  limited  to  areas 
directly  related  to  delivery  of  information  and  the  acquisition  of  knowledge. 

Management  of  student  flow  in  the  PLATO  portion  of  training  developed  by  a process  of  evolution. 
Many  traditional  teaching  methods  were  employed  and  it  appeared  that  little  in-depth  examination  of 
optimum  student  management  techniques  was  carried  out.  The  course  was  operated  under  the  assumption 
that  PLATO  would  not  reduce  instructor  requirements  and,  therefore,  the  system  should  be  utilized  only  in 
ways  that  fit  into  traditional  modes  of  instruction.  It  was  impossible  to  achieve  major  cost  savings  through 
PLATO  applications  while  this  attitude  was  present. 

Instructor  Role  in  Pl.ATO-Based  Training.  Tire  role  of  instructors  in  the  new  PLATO  environment, 
like  management  of  student  flow,  also  took  shape  through  an  evolutionary  process.  The  instructor  tended 
to  become  less  of  a dispenser  of  information  and  more  of  a facilitator  of  instruction.  For  the  most  part,  the 
role  was  primarily  one  of  dealing  with  uncertainties  including:  how  long  it  would  take  to  get  through  a 
lesson:  whether  enough  terminals  would  be  available  when  needed;  if  sufficient  ECS  would  be  provided  to 
support  all  of  the  lessons  as  needed;  whether  the  system  would  be  operating;  and,  if  it  would  fail  during  an 
on-line  session  making  it  necessary  to  revert  to  backup  material.  With  this  substantial  amount  of 
unpredictability  facing  the  instructor  each  day,  his  management  role  in  making  on-the-spot  adjustments  was 
critical.  Instructors  were  not  only  responsible  for  the  direct  instructional  processes  associated  with 
monitoring  the  delivery,  acquisition,  and  evaluation  of  learning,  but  also  for  indirect  processes  to  establish  a 
learning  environment. 

Evaluating  this  role  could  not  be  done  realistically  without  considering  characteristics  of  PLATO.  If 
the  system  were  perfectly  reliable,  ample  ECS  were  always  available,  and  terminals  never  failed,  many  of 
the  contingencies  facing  instructors  would  not  have  existed.  In  this  case,  less  of  a management  load  would 
have  been  levied  on  instructors  and  more  time  would  have  been  spent  in  direct  instructional  activities. 
During  the  course  of  the  project,  however,  the  instructor  role  continued  to  incorporate  a great  deal  of 
uncertainty  management.  This  was  a negative  aspect  of  the  PLATO  system,  but  ready  solutions  were  not 
apparent. 

Instructor  duties  in  the  PLATO  lab  were  very  different  from  the  conventional  classroom.  PLATO 
students  dealt  with  a far  more  versatile  and  complex  teaching  medium  than  textbooks  or  lectures.  PLATO 
lessons  included  remedial  loops,  branches  to  HELP  sequences,  conditional  answers,  and  sequential 
animations.  If  a student  asked  for  help  in  a particular  lesson,  the  instructor  was  required  to  determine 
where  the  student  had  been,  how  he  had  arrived  at  his  difficulty,  and  how  to  help  him  in  the  most 
instructionally  effective  way.  To  do  this  efficiently,  the  instructor  had  to  be  intimately  familiar  with  all  of 
the  PLATO  lessons;  not  just  lesson  content,  but  with  the  strategies  used,  answer  idiosyncracies,  rationales 
and  logical  structures.  Being  thorou^ily  familiar  with  even  the  limited  number  of  PLATO  lessons  involved 
in  this  service  test  was  a major  instructor  accomplishment.  The  instructor  task  was  further  complicated  by 
the  fact  that  many  of  the  PLATO  lessons  were  not  designed  for  “browsing.”  That  is,  there  was  often  no 
simple  way  to  go  back  to  a previous  display  or  otherwise  reconstruct  the  student’s  progression.  Such 
options  are  not  precluded  by  TUTOR  software,  but  are  complex  to  code  and  were  generally  not  employed 
in  the  interest  of  coding  efficiency  and  rapid  lesson  development. 

The  job  of  the  instructor  in  the  classroom  was  made  very  difficult  by  such  lesson  inflexibility.  While 
expertise  in  anticipating,  comprehending,  and  correcting  student  difficulties  came  with  time  and 
experience,  new  instructors  were  at  a severe  disadvantage.  These  considerations  pointed  out  a strong  need 
to  work  toward  making  PLATO  lesson  material  self-supporting  so  that  instructor  intervention  would  be 
kept  to  a minimum.  This  would  require  a long  period  of  continuous  evaluation  and  revision  as  lessons  are 
used  in  operational  training. 

Evaluation  and  revision  of  PLATO  lesson  material  created  another  facet  of  the  instructor  role.  As 
instructors  used  the  lessons,  they  noticed  areas  where  students  had  consistently  encountered  difficulties. 
The  instructors  brought  this  information  to  the  author  group  for  corrective  action.  When  the  lessons  were 
first  introduced,  there  was  a great  deal  of  such  feedback.  Each  instructor  had  different  ideas  on  how  lessons 
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should  be  improved.  Some  suggestions  entailed  substantial  improvements  in  efficiency,  while  others  were 
fairly  trivial  and  dealt  with  minor  word  changes,  additions,  or  rearrangements  that  were  concerned  more 
with  aesthetics  than  lesson  content.  Instructors  went  directly  to  the  author  group  with  their  comments  and 
left  the  decision  as  to  whether  or  not  to  incorporate  the  comments  to  the  authors.  The  number  of 
comments  was  so  great  that  it  would  have  been  impossible  to  respond  to  each.  To  alleviate  this  burden  for 
the  authors,  a procedure  was  established  whereby  instructors  took  their  comments  to  their  supervisor,  who 
was  expected  to  forward  them  to  the  authors.  This  procedure  resulted  in  an  instant  bottleneck  for 
instructor  feedback  on  the  lessons.  When  instructors  had  a comment,  they  had  to  explain  where  a change 
should  be  made  in  the  lesson.  The  supervisor,  not  intimately  familiar  with  the  content  of  all  the  lessons, 
usually  had  to  be  shown  what  the  instructor  was  trying  to  change  and  then  would  have  to  translate  the 
request  to  the  author  group. 

Introducing  a supervisor  into  the  link  between  instructors  and  authors  created  a subtle  barrier 
between  them.  Instructors  became  more  distant  from  the  lessons  they  worked  with  daily.  As  a result,  one 
of  the  better  features  of- PLATO,  the  ability  to  rapidly  change  or  alter  lesson  material,  was  pushed  into  the 
background  and  its  benefits  diminished.  Furthermore,  it  encouraged  instructors  to  accept  undesirable 
characteristics  in  lessons  rather  than  to  struggle  with  changing  them.  The  net  effect  of  the  procedure  was 
that  lessons  never  reached  levels  of  self-sufficiency  that  would  reduce  or  eliminate  needs  for  instructor 
„ intervention  in  PLATO  training. 

A development  in  systems  software  enabled  a great  deal  of  progress  to  be  made  in  solving  the  lesson 
revision/feedback  problem.  The  new  software  change  allowed  a student  to  write  a note  about  a particular 
lesson.  The  note  was  stored  in  a file  which  could  then  be  accessed  by  any  author  or  instructor  in  the  course. 
The  students  could  also  review  notes  they  had  written,  as  well  as  notes  written  by  other  students  or 
instructors.  When  a student  wrote  a note,  the  name  of  the  lesson  and  location  within  the  lesson  were 
displayed  in  the  note.  This  feature  greatly  facilitated  locating  and  correcting  troubleshome  areas  within 
lessons.  For  CAI  to  work  effectively,  it  will  be  necessary  to  emphasize  to  students  and  instructors  alike  that 
the  comments  feature  is  available  and  that  it  should  be  used  often. 

A fourth  aspect  of  the  instructor  role  in  PLATO  training  concerns  control  of  student  learning 
strategies.  It  was  an  important  function  of  instructors  to  insure  that  students  applied  proper  reading  and 
study  techniques  to  acquire  knowledge  in  their  independent  learning  activities.  One  area  some  instructors 
emphasized  for  their  students  was  that  of  note  taking,  since,  students  had  no  way  to  review  PLATO 
material  outside  of  the  lab.  To  remedy  this  problem,  some  ii.„tructors  insisted  that  students  take  notes 
while  going  through  each  lesson.  This  strategy  imposed  some  unusual  considerations  for  computer-based 
training. 

Most  computer-based  lessons  were  designed  and  validated  in  such  a way  that  students  were  expected 
to  be  able  to  pass  an  end-of-lesson  test  solely  on  the  basis  of  knowledge  gained  as  a result  of  interacting 
with  the  lesson.  Introducing  note  taking  allowed  students  to  write  down  information  they  were  supposed  to 
have  stored  in  memory.  When  the  end-r>f-lesson  test  was  encountered,  students  had  their  notes  to  refer  to 
for  assistance  on  each  question.  The  validity  of  the  test  may  have  been  compromised,  as  it  did  not 
necessarily  measure  retention  so  much  as  it  did  note  taking  ability. 

A possible  solution  to  this  problem  was  to  design  lessons  with  the  thought  in  mind  that  students 
would  be  taking  and  using  notes  throughout  the  lesson  and  end-of-lesson  test.  The  author  could  then  create 
interactions  with  the  lesson  that  required  students  to  synthesize  facts  and  principles  which  could  be 
recorded  in  notes,  but  which  had  to  be  combined  to  satisfy  response  requirements.  Likewise,  end-of-lesson 
test  questions  could  be  carefully  constructed  so  that  they  do  not  require  responses  that  are  simply  verbatim 
reproductions  of  information  presented  in  the  lesson. 

Such  a practice  would  be  more  congruent  with  what  a student  must  do  on  the  job  after  training.  It  is 
rare  that  an  individual  in  the  field  is  faced  with  a problem  that  he  must  solve  without  access  to  some  form 
of  written  material.  This  may  also  prove  to  be  a more  effective  learning  technique  than  study  without  notes 
for  it  would  require  deeper  thought  and  actual  understanding  than  is  required  by  simple  verbatim 
reproduction  of  facts. 


72 


In  summary,  it  was  observed  that  instructors  served  a very  important  function  as  managers  in  the 
daily  performance  of  their  training  responsibilities.  Their  jobs  were  complicated  by  inflexibilities  in  lessons 
and  by  requirements  to  be  intimately  familiar  with  lesson  content  and  structure.  They  provided  authors 
with  information  to  improve  lessons,  but  inefficient  communication  hampered  this  activity.  Finally, 
instructor  control  of  student  study  strategies  and  note  taking  was  observed  to  complicate  aspects  of  lesson 
validity  and  effectiveness.  Allowing  note  taking  could,  nevertheless,  be  a powerful  technique  for  improving 
training  effectiveness. 

In  general,  the  instructor  role  was  still  evolving  during  the  preparation  of  this  report.  It  was  not 
well-defined  and  will  require  additional  experience  before  a truly  summative  assessment  can  be  made.  As 
the  system  continues  to  develop,  efforts  should  be  directed  toward  increasing  instructor  involvement  in 
direct  instructional  support,  reducing  requirements  for  instructor  intervention  in  learning  from  PLATO 
material,  and  increasing  involvement  in  lesson  refinement. 

Management  of  Student  Testing.  This  section  deals  with  use  of  PLATO  for  on-line  administration, 
scoring,  and  critique  of  the  end-of-block  examination,  the  principal  tool  for  evaluating  student  learning. 
Occurring  at  the  end  of  each  major  section  of  training,  the  end-of-block  examination  was  20-  to  50- item 
multiple-choice  test,  designed  to  sample  the  student’s  knowledge  of  the  subject  matter  covered.  The  test 
was  taken  by  the  class  as  a group  and  use  of  notes  was  prohibited.  There  were  three  alternate  forms  of  each 
test  with  a controlled  overlap  of  items  that  were  common  to  all  forms.  All  end-of-block  examinations  were 
administered  on  the  PLATO  system  to  all  classes  for  each  block,  including  the  shred  area  blocks  where  no 
actual  instruction  was  presented  on  PLATO. 

Access  to  PLATO-administercd  tests  was  controlled  by  the  instructor.  When  a class  was  ready  to  take 
a block  examination,  the  instructor  entered  a particular  test  form  for  each  student.  All  students  in  a class 
received  the  same  form  of  the  test.  The  instructor’,  guidance  was  to  select  the  form  that  had  been  used 
least.  The  instructor  went  to  each  student’s  terminal  and  entered  a code  (which  the  student  did  not  see) 
which  “unlocked”  the  test  and  allowed  the  student  to  complete  it.  After  a student  had  gone  through  all  of 
the  items  on  his  test,  he  was  given  the  option  of  either  having  it  graded  or  going  back  and  making  desired 
changes  to  any  of  his  answers.  Once  satisfied  with  his  answers,  he  had  the  test  graded.  An  instructor  then 
assisted  students  individually  as  each  missed  item  was  reviewed.  Students  who  had  completed  the  test  were 
required  to  wait  until  the  instructor  was  free  to  complete  the  grading  and  review  procedure. 

This  method  of  testing  proved  to  be  effective,  although  less  efficient  than  may  have  been  possible.  It 
was  less  time-consuming  than  traditional  papcr-and-pencil  testing  and  conserved  materials.  Perhaps  the  only 
disadvantage  was  the  critique  procedure.  Since  test  items  were  not  discussed  in  a group  situation,  students 
who  guessed  answers  correctly  missed  a discussion  of  why  the  answer  was  right. 

Management  of  the  on-line  testing  material  seemed  somewhat  less  than  optimal.  A more  efficient  use 
of  PLATO’s  power  could  reduce  the  instructor  activity  presently  required  to  prepare  for  testing  sessions. 
An  algorithm  could  be  designed  to  automatically  output  on  any  given  day  the  least  used  test  form.  As  long 
as  a secure  “lock-out”  code  had  to  be  entered  before  the  file  could  be  used,  students  would  not  be  able  to 
get  premature  access  to  end-of-block  tests. 

Test  creation,  revision  and  security  were  generally  facilitated  by  PLATO.  For  example,  a new  test 
could  oc  ready  for  administration  a few  hours  after  the  test  items  were  approved.  The  items  were  typed 
into  designated  locations  within  the  TUTOR  code  which  was  reproduced  for  each  new  test.  This  saved  the 
time  tag  and  additional  efforts  associated  with  off-line  reproduction.  In  addition,  revision  of  tests  was 
greatly  simplified,  since  changes  could  be  made  without  the  same  time  lags  and  clerical  functions  associated 
with  off-line  testing.  Revision  was  further  facilitated  by  providing  a subroutine  which  allowed  instructors  to 
enter  comments  about  items  which  were  ambiguous,  in  error,  or  otherwise  caused  problems  for  their 
students.  These  comments  were  then  reviewed  by  the  supervisory  staff.  Security  of  test  materials  was  also 
facilitated  by  the  absence  of  a need  to  lock  them  in  a file  cabinet  to  be  checked  out  for  use  and  checked  in 
afterwards.  Instructors  did  not  have  to  visually  inspect  each  page  of  each  test  for  pencil  marks  left  by  the 
student.  A printed  copy  of  each  block  test  was  maintained  by  the  course.  This  copy  was  initialed  by  a 
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member  of  the  Instruction  and  Measurement  unit,  and  all  items  on  the  PLATO  version  of  the  test  had  to 
match  this  hard  copy.  If  a minor  change  were  to  be  made  in  a particular  item,  it  was  entered  in  pen-and-ink 
on  the  hard  copy  which  was  sent  for  approval.  If  approved,  the  change  was  initialed  and  returned  to  the 
course.  A PLATO  author  then  took  action  to  change  the  coded  item  in  the  system.  Tests  were  periodically 
reviewed  to  insure  that  coded  items  matched  the  approved  hard  copy. 

Test  item  analyses  paralleled  ATC  procedures,  except  that  they  were  performed  by  PLATO 
personnel.  These  analyses  provided  a rough  measure  of  the  reliability  and  validity  of  the  test  and  its  items. 
What  the  instructor  viewed  on  PLATO  was  a representation  of  the  test  analytic  forms  which  were  once 
prepared  manually.  Test  evaluation  could  be  improved  via  the  PLATO  system  since  the  data  collection  and 
computational  aspects  of  the  task  could  be  automatic.  More  sophisticated  test  analysis  is  possible,  as  long  as 
instructors  are  trained  to  properly  interpret  the  results. 

In  summary,  management  of  on-line  testing  had  advantages  and  disadvantages,  although  its  advantages 
appeared  to  be  more  numerous.  It  is  likely  that  management  of  tests  has  perhaps  become  as  efficient  as 
possible  under  the  current  circumstances,  with  the  exception  of  providing  student  access  to  tests  when 
needed.  It  is  recommended  that  further  attention  be  given  to  reducing  the  need  for  instructors  to  manually 
select  and  set  up  tests  for  student  use. 

General  Conclusions  Concerning  Management 

This  section  examines  various  aspects  of  the  management  portion  of  PLATO-based  operational 
training.  Management  activities  in  daily  operations,  procedures  for  controlling  student  flow,  the 
management  of  instructional  material,  instructor  roles,  and  testing  were  each  discussed  in  detail  relative  to 
unique  contributions  of  PLATO.  While  many  benefits  were  provided  by  PLATO  in  each  of  these  areas, 
there  were  certain  disadvantages.  On  the  whole,  however,  it  was  observed  that  PLATO  did  not  lead  to 
insurmountable  management  problems.  With  continual  interest  in  smoothing  and  streamlining  the  PLATO 
operation  and  reducing  the  need  for  “hand  feeding”  it,  increased  training  program  efficiency  is  possible. 

With  the  relatively  limited  experience  available  in  managing  a complex  medium  such  as  PLATO, 
management  was  remarkably  good.  Management  of  PLATO-based  training  in  Air  Force  technical  training 
did  not  entail  unusual  or  drastic  measures  to  function.  While  different  techniques  were  necessary  to  allow 
for  PLATO’s  idiosyncracies.  these  techniques  served  to  facilitate  efficient  training  rather  than  impede  it. 
Keys  to  success  in  future  Air  Force  applications  are  contained  in  the  experience  gained  from  this  effort.  For 
example,  to  insure  success,  it  is  necessary  to  engage  in  substantial  preplanning  and  possess  a willingness  to 
depart  from  traditional  methods  of  training  management.  With  such  an  approach,  implementation  of 
computer-based  concepts  into  technical  training  should  be  carried  out  with  increasingly  greater  efficiency. 


VIII.  HUMAN  FACTORS  ANALYSIS 

Introduction  and  Method 

The  PLATO  system  was  analyzed  from  human  factors  engineering  standpoints  of  usage  problems  and 
parameters,  reliability,  student  courseware  adaptation,  hardware  familiarization,  and  TUTOR  coding 
difficulties.  For  the  sake  of  convenience,  the  methodology  for  each  human  factor  standpoint  is  included 
within  the  results  and  discussion. 

Awlt*  and  Discussion 

Usage  Problems  and  Parameters.  Student  usage  problem*  were  measured  in  several  ways:  (a) 
altitudinal  questionnaires  administered  1 to  3 days  prior  to  the  Mock  examinations  in  the  first  four  blocks, 
(b)  a “Human  Factors  Questionnaire”  developed  by  the  evaluation  team  given  midway  (approximately  2 
weeks)  into  block  II,  and  (c)  “Student  Critique”  forms  given  at  the  completion  of  block  IV  and  at  course 
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completion-  Ongoing  observation  by  evaluation  team  members,  as  well  as  continuous  feedback  from 
instructors  and  managers  of  the  PLATO  IV  system,  supplemented  these  primary  data. 

The  data  revealed  few  usage  problems  with  the  major  components  of  the  PLATO  IV  system. 
Approximately  90  percent  of  the  respondents  to  the  Human  Factors  Questionnaire  indicated  no  major 
problems  with  the  use  of  the  keyboard  or  reading  the  plasma  panel.  The  remaining  comments,  such  as  the 
seating  arrangements,  terminal  position,  system  reliability,  lighting,  glare,  and  microfiche  resolution  (color 
and  focus),  were  not  mentioned  with  sufficient  frequency  to  warrant  further  examination.  In  part,  this  was 
due  to  the  existing  PLATO  laboratoiy  environment.  The  26-  by  50-foot  laboratory  space  was  lighted  by 
three  blue  fluorescent  lamps,  and  a small  variable-intensity  lamp  was  located  in  each  of  the  terminal  carrels. 
Thus,  sufficient  light  was  available  for  note  taking,  while  reflective  glare  was  minimized.  There  were  some 
instructor  complaints  about  the  blue  lights  in  the  room;  consequently,  a better  solution  to  the  glare 
problem  would  be  diffused  lighting. 

Attitudes  concerning  the  optimal  length  of  PLATO  sessions  were  obtained  by  using  an  open-ended 
question  on  the  Human  Factors  Questionnaire.  The  average  length  recommended  was  approximately  60 
minutes,  with  responses  ranging  from  10  minutes  to  2 hours.  Approximately  10  percent  wanted  to 
individually  set  session  time  according  to  need.  The  main  factor  determining  desired  session  time  before 
breaks  appeared  to  be  tradition,  since  60  minutes  is  the  normal  amount  of  time  spent  in  a classroom  before 
a break.  Eyestrain  and  headaches  were  additional  factors,  with  10  percent  of  the  respondents  indicating 
that  using  PLATO  beyond  60  minutes  was  not  desirable. 

PLATO  System  Reliability.  The  relationship  between  the  student  and  the  PLATO  medium  is  most 
obviously  influenced  by  the  reliability  of  the  PLATO  system.  CERL  maintained  records  and  this 
information  is  presented  in  Table  32.  The  data  basically  reflect  the  service  test’s  second  phase,  when 
development  tryout  and  evaluation  activities  were  ongoing.  Most  interruptions  were  short  and  more 
frustrating  than  representive  of  a real  problem.  Overall,  terminal  and  system  reliabilities  were  good  during 
the  latter  portion  of  the  service  test.  Terminals  which  failed  were  usually  repaired  within  1 to  2 days. 


Table  32.  PLATO  Reliability  Information 


Intarruptlons 

Mean  P*rc«flt 

Otabla  Tima 

Main  Hours 

Batwaan  Intarruptlom 

Mun  Hours 
Duration 

94.7 

PLATO  System 

9.52 

.38 

Main  Waofcs  BitwNn 

Maan  Hours 

Unavailabla* 

Terminal  Failures 

11.31 

Terminals 

7.02  (per  terminal/per  month) 

includes  time  when  no  maintenance  was  available  (nights  and  weekends). 


Student  Adaptation.  Student  adaptation  to  CAI  learning  strategies  was  measured  by  an  open-ended 
question  on  the  Human  Factors  Questionnaire  as  well  as  by  periodic  student  interviews.  The  results 
appeared  to  suggest  no  significant  adaptability  problems  with  using  the  PLATO  IV  lessons.  Of  the  76 
students  in  the  CP  and  PB  conditions  who  responded  to  the  Human  Factors  Questionnaire,  approximately 
91  percent  indicated  that  it  did  not  take  long  to  get  used  to  the  PLATO  lessons.  Of  the  few  students  who 
indicated  adaptability  problems,  most  were  related  to  the  remedial  branching  strategy  of  the  lessons  (being 
“kicked  back”  in  the  lesson  for  missing  a question)  or  to  signing-on  difficulties,  both  of  which  were  found 
to  be  relatively  unimportant.  Periodic  student  interviews  confirmed  these  findings. 
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Terminal  and  Keyboard  Familiarization.  Terminal  and  keyboard  familiarization  were  measured  with 
an  open-ended  question  on  the  Human  Factors  Questionnaire.  The  majority  of  respondents  indicated  that 
orientation  to  PLATO  IV  did  not  take  very  long.  Of  the  76  usable  responses,  the  mean  time  indicated  to 
become  used  to  the  terminal  was  approximately  2 hours,  with  a wide  range  of  3 minutes  to  2 weeks.  Only 
two  respondents  indicated  they  had  not  become  used  to  the  terminal.  There  were  a few  comments  on  how 
relatively  simple  and  easy  it  was  to  use,  and  that  prior  typing  experience  had  helped  to  speed  up  the 
familiarization  process. 

TUTOR  Coding  Difficulties.  Information  on  this  topic  was  obtained  from  the  instructional  materials 
developments  interviews  conducted  in  November  1974.  The  results  showed  that  authors  had  no  major 
problems  in  manipulating  the  terminal  for  lesson  coding  activities.  One  problem  was  that  the  keyboard  did 
not  indicate  all  the  various  functions.  For  example,  an  author  had  to  remember  the  keys  which  had  to  be 
pressed  to  obtain  special  characrers.  In  this  case  the  author  could  create  his  own  chart  of  characters  and 
functions,  or  access  a lesson  which  described  them. 

Conclusions 

This  investigation  into  the  human  factors  consideration  of  a military  PLATO  site  found  no  major 
human  factors  problems  at  Chanute’s  PLATO  laboratory. 


IX.  EXPERIMENTAL  STUDIES 

ARPA  supported  an  extension  of  the  PLATO  IV  service  test  at  Chanute  so  that  sufficient  time  to 
fully  implement  and  evaluate  the  PLATO-based  Special  Purpose  Vehicle  Repairman  courses  was  available. 
Additional  objectives  were  specified  in  the  revised  management  plan  (see  historical  perspecrive,  Phase  III). 
One  of  the  objectives  included  the  performance  of  seven  studies  which  were  directed  at  examining  PLATO’s 
potential  in  selected  CAI  strategies,  and  CMI  applications.  Due  to  author  attrition,  managerial  problems,  the 
fact  that  the  collective  ability  of  the  remaining  authors  could  not  meet  the  demands  of  the  task,  and  other 
commitments,  only  one  study  was  completed. 

The  study  attempted  involved  the  development  of  a lesson  dealing  with  the  Air  Force  technical  order 
system.  Technical  Orders  (TO)  had  been  a difficult  subject  area  for  technical  training  students  because  they 
had  little  background  information  with  which  to  associate  the  new  material.  Moreover,  TOs  are  a 
low-interest  subject  area.  It  was,  however,  critical  that  students  learned  to  rely  on  TOs,  which  were  the  only 
official  guidance  for  conducting  maintenance  activities.  Through  the  use  of  CAI  simulation  and  gaming 
instructional  strategies,  it  was  expected  that  greater  learning  efficiency  and  positive  affect  would  result. 

The  final  TO  instructional  sequence  deviated  from  the  original  design  in  that  the  simulation  exercise 
was  not  completed,  and  that  the  gaming  activities  were  curtailed  and  modified.  The  aforementioned 
problems  which  reduced  the  scope  of  activities,  were  responsible  for  these  deviations. 

This  section  of  the  report  will  present  findings  derived  from  the  evaluation  of  the  TO  instructional 
sequence.  At  best,  it  could  be  called  a pilot  study  since  operational  constraints  weakened  the  evaluation 
design  considerably. 

Lesson  Sequence  and  Development  Time 

The  TO  instructional  sequence  consisted  of  seven  modules,  each  with  its  own  test.  Students  had  to 
attain  a minimum  score  of  80  percent  for  each  module  test  to  proceed  to  the  next  module.  Five  games  were 
prepared  and  were  included  in  the  instructional  sequence  for  some  of  the  students  in  the  vehicle 
maintenance  courses.  Figure  1 2 depicts  the  instructional  sequence  and  the  points  at  which  the  games  were 
added  to  the  lesson. 

The  five  games  were  intended  to  keep  the  student  involved  with  TO  material  and  to  promote  a 
positive  attitude  toward  subject  matter  and  learning  environment.  It  was  expected  that  learning  acquisition 
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and  retention,  task  performance,  and  the  attitudinal  measures  taken  for  each  student  would  indicate  an 
improvement  over  the  leasson  versions  without  gaming.  Two  of  the  games  offered  competition  between  the 
student  and  PLATO  (timed  games)  while  three  offered  competition  between  students.  A brief  description 
of  the  five  games  is  given  as  follows: 

1.  TIC  TAC  TO  - One  student  challenged  another  who  was  playing  the  game.  Like  tic-tac-toe,  the 
object  was  to  get  three  Xs  or  Os  in  a row.  Student  selected  a square  and  had  to  answer  a multiple-choice 
question  correctly  to  have  his  sign  placed  in  the  square  (Figure  1 3). 

Development  Time  On-Line  Off-Line 

(in  hours)  47  26 

2.  BROWN  BRAIN  - The  student  is  given  about  5 seconds  to  identify  a technical  order  number 
generated  by  PLATO.  He  selects  the  type  of  TO  by  touching  a box  containing  the  name  (Figure  14). 

Development  Time  On-Line  Off-Line 

(in  hours)  57  28 

3.  QUICK  DRAW  — One  student  challenged  another  student  within  the  game  to  a gunfight. 
Students  took  turns  answering  multiple-choice  questions  to  draw  guns  and  fire. 

Development  Time  On-Line  Off-Line 

(in  hours)  121  33 

4.  ANXIETY  - The  student  is  given  120  seconds  to  select,  in  order,  the  steps  necessary  to  locate 
information  in  one  of  several  Technical  Order  types  (Figure  15). 

Development  Time  On-Line  Off-Line 

(in  hours)  45  20 

5.  W1ZZARD  - The  student  moves  around  a game  board  to  acquire  wealth  and  power.  He  must 
periodically  answer  questions  in  order  to  proceed  (Figure  16). 

Development  Time  On-Line  Off-Line 

(in  hours)  170  42 

On-line  coding  of  these  games  was  beyond  the  capabilities  of  site  personnel  because  the  computer 
programmer  was  not  as  yet  proficient  with  the  TUTOR  language.  Of  the  430  hours  expended  to  code  the 
lessons,  244  hours  were  obtained  from  outside  the  immediate  project.  Outside  sources  also  contributed  73 
hours  of  the  off-line  development  total  of  163  hours. 

According  to  site  records,  586  on-line  and  587  off-line  hours  were  required  to  complete  and  validate 
the  seven  TO  modules.  These  figures  are  somewhat  inflated  because  of  a management  change,  the  loss  of 
the  author  originally  responsible  for  development,  failure  to  follow  the  original  concept,  and  numerous 
other  difficulties.  Total  lesson  development  time  inclusive  of  games  was  about  1 ,800  hours. 

Method 

Evaluation  Design.  The  evaluation  design  employed  three  conditions.  A non-PLATO  (NP)  or  control 
condition  consisted  of  the  instructional  sequences  in  use  at  the  time  of  the  study.  Since  Special  Purpose  and 
General  Purpose  Vehicle  Maintenance  TO  sequences  were  somewhat  similar,  all  vehicle  maintenance 
students  were  used  in  the  study.  The  NP  condition  actually  consisted  of  two  control  groups,  one  for  each 
course.  This  was  necessary  because  of  the  differences  in  the  instructional  sequences.  Though  course 
objectives  were  virtually  identical,  the  General  Purpose  Vehicle  Maintenance  student  received  a lecture 
discussion  presentation  in  9 hours,  while  the  Special  Purpose  Vehicle  Maintenance  student  completed  the 
material  in  about  13  hours  with  the  principal  media  being  programmed  texts  and  workbooks.  A second 
condition  was  termed  the  PLATO  version  (PV).  It  consisted  of  the  PLATO  lessons  without  the  games,  and 
presented  background  knowledge  and  procedural  steps  for  locating  technical  data  related  to  vehicle 
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Figure  13.  TIC  TAC  T.O.  game. 
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Figure  15.  Anxiety  game. 


0 f'  f ens  i ve  Spe  11s 

ill  Sleep  (will  weaken  other  wis.) 


Freeze  (can't  move) 


Blast  Bolt  (to  kill) 


De  f ens i ve  Spe 11s 
OFF  Invisibility 
OFF  Count er  I nv i s i b 1 . 

OFF  Tel epor t at i on 

OFF  Peek  (opponent ' s data) 

TO  MOVE:  arrow  keys 
USE  SPELL:  first  letter  of 
t hat  sp>e  1 1 {ex . " s " si eep , 

"h"  hex,  etc..} 


Figure  16.  Wizzard  game. 


Rga i nst  YOU 


maintenance.  The  remaining  condition,  adjunct  gaming  (AG),  combined  the  five  PLATO  games  with  the 
basic  PLATO  lessons.  Students  in  the  General  Purpose  and  Special  Purpose  Vehicle  Repairman  courses  were 
assigned  to  the  PV  and  AG  conditions. 

Measures.  Performance  measures  included  lesson  completion  time,  end-of-lesson  test  scores,  and  a 
specially  constructed  TO  topical  test.  Items  were  prepared  by  selecting  subject  areas  that  were  common  to 
all  lesson  versions,  and  insuring  that  the  items  would  not  favor  a particular  lesson  version.  The  final  version 
of  the  TO  test  consisted  of  39  multiple-choice. items,  after  numerous  revisions  to  enhance  reliability. 

The  attitude  questionnaire  consisted  of  four  a priori  dimensions  (four  items  each):  TO  lesson 
effectiveness,  gaming  affect,  competition,  and  TO  subject  matter.  A 9-point  ( 0 = Strongly  Disagree  to  9 = 
Strongly  agree)  Likert-type  scale  was  used  for  each  item. 

Subjects.  A total  of  79  General  Purpose  and  Special  Purpose  Vehicle  Maintenance  students 
participated.  Table  33  gives  a course  by  condition  breakdown.  No  Special  Purpose  Vehicle  Maintenance 
students  were  obtained  for  the  NP  condition,  and  numerous  constraints  account  for  missing  observations  in 
subsequent  tables. 


Table  33.  Sample  Breakdown  for  TO 
Lesson  Evaluation 


Condition 

Court* 

N 

Non  PLATO  (NP) 

General  Purpose 

31 

Special  Purpose 

0 

PLATO  Version  (PV) 

General  Purpose 

11 

Special  Purpose 

12 

Adjunct  Gaming  (AG) 

General  Purpose 

15 

Special  Purpose 

10 

Procedures.  Slightly  different  procedures  were  followed  for  each  condition.  In  the  NP  condition, 
students  received  the  normal  instructional  program  and  then  received  the  TO  test  and  attitude 
questionnaire.  PV  students  received  the  PLATO  lesson  in  lieu  of  the  programmed  text  on  TOs.  AG  students 
received  the  seven  CAI  lesson  modules,  but  played  a game  prior  to  taking  the  end-of-lesson  test  for  those 
modules  which  contained  games.  If  a game  required  a student  competitor,  and  no  other  students  were 
presently  within  the  module,  an  instructor  acted  as  the  opponent.  Data  collection  for  end-of-lesson  test 
scores,  lesson  times,  and  times  within  games  were  automatically  accomplished  via  PLATO,  for  the  AG  and 
PV  conditions. 

Procedures  differed  between  courses  as  well.  General  Purpose  Vehicle  Maintenance  students  received 
the  TO  lesson  sequence  at  the  end  of  their  first  instructional  block,  while  the  Special  Purpose  Vehicle 
Maintenance  students,  after  an  introductory  lesson  to  the  PLATO  system,  received  their  lesson  as  the  first 
in  the  course.  Test  and  questionnaire  administration  times  were  approximated  1 day  apart  for  the  courses. 
General  Purpose  Vehicle  Maintenance  students  were  required  to  take  block  tests  before  they  received  the 
evaluation  instruments. 

Results 

Lesson  Performance  Findings.  Lesson  module  mean  completion  times  and  scores  are  presented  in 
Table  34.  Mean  game  completion  times  are  also  included.  Accurate  lesson  completion  times  for  the  NP 
condition  were  not  obtained;  however,  it  was  estimated  that  General  Purpose  Vehicle  Maintenance  students 
were  given  about  330  minutes  to  leam  the  same  material.  As  constraints  (i.e.,  low  student  load)  prevented 
the  assignment  of  any  Special  Purpose  Vehicle  Maintenance  students  to  the  NP  condition,  no  data  were 
obtained.  All  NP  condition  students  were  from  the  General  Purpose  Vehicle  Repairman  course. 
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Table  34.  Mean  TO  Lesson  Module  Times  and  Scores 


r 


Lesson  Module 
(June) 

Time  (Minutes) 

Score  (Percent) 

PLATO  Version 

N = 21 

Adjunct  Gaming 

N = 16 

PLATO  Version 

N = 21 

Adjunct  Gaming 
N » 16 

1. 

Introduction 

47.85 

34.00* 

72.06 

75.88 

2. 

Types  of  TOs 

54.23 

50.68 

80.53 

78.24 

(Tic  Tac  TO) 

(24.7)a 

3. 

TO  Numbering 

47.40 

57.29 

83.18 

84.11 

(Brown  Brain) 

(9.8) 

4. 

Locating  an  Index 

52.41 

69.12 

65.29 

73.12 

5. 

Locating  TOs 

34.19 

35.94 

72.38 

68.82 

(Quick  Draw) 

(10.8) 

6. 

Using  the  I.P.B. 

37.36 

37.06 

80.50 

76.87 

(Anxiety) 

(8.0) 

7. 

Updating  TOs 

24.90 

22.88 

81.09 

75.88 

(Wizzard) 

(24.0) 

Total 

298.34 

306.97 

(77.3) 

aMean  game  time  is  not  included  in  the  mean  lesson  time  for  adjunct  gaming  condition. 
*p  < .05. 


Each  set  of  module  completion  times  for  the  PV  and  AG  conditions  was  analyzed  through  ANOVA 
techniques.  Only  the  first  module  had  a significant  difference  (F(l,35)  = 5.20,  p < .05).  The  mean  total 
completion  times  for  students  in  either  condition  were  about  300  minutes  or  5 hours,  with  the  inclusion  of 
gaming  strategies  adding  1 hours  and  15  minutes. 

Module  scores  were  analyzed  in  a similar  fashion.  There  were  no  important  differences  for  any  of  the 
seven  sets  of  module  scores,  or  the  average  of  module  scores. 

When  administered  in  the  actual  experiment,  the  39-item  test  of  student  achievement  had  an 
estimated  reliability  of  .63;  therefore,  five  items  were  deleted,  increasing  Cronbach’s  alpha  to  .68.  A 
one-way  analysis  of  variance  was  performed  on  the  raw  scores  with  conditions  as  the  independent  variable 
(F  (2,76)  = 4.34,  p < .05).  The  least  significant  difference  (LSD)  multiple  comparison  method  showed  that 
the  mean  raw  score  for  the  PV  condition  was  significantly  greater  than  the  NP  condition,  (Af^p  = 16.77, 
Afpv  = 20.26,  M\c,~  1 9.24;  LSD=  2.82,  p < .05).  Moreover,  the  combined  PLATO  means  were  also 
significantly  different. 

Attitudinal  Findings.  The  internal  consistency  of  the  four  dimensions  was  estimated  using  Cronbach’s 
alpha.  Table  35  indicates  that  all  but  the  competition  scale  had  adequate  reliability.  The  competition  scale 
was,  therefore,  dropped  from  further  consideration  for  the  remaining  scales.  A scale  score  was  computed  by 
summing  individual  item  responses  and  dividing  by  the  number  of  items.  Scale  means  are  reported  in  Table 
36.  The  attitudinal  scales  were  constructed  so  that  a score  of  4.5  constituted  a neutral  point  with  higher 
favorability  being  indicated  by  a higher  score.  An  analysis  of  variance  was  performed  on  student  scale  scores 
with  condition  as  the  independent  variable.  No  significant  differences  were  found  for  perceived  lesson 
effectiveness  or  gaming  affect  using  the  .05  level  of  confidence.  From  the  ANOVA  results  (Table  37),  it  can 
be  seen  that  mean  scores  for  the  PV  condition  were  significantly  higher  than  those  of  the  other  conditions 
on  the  technical  order  affect  scale. 

Discussion 

The  findings  suggested  that  gaming  had  little  effect  on  student  performance  and  attitudes.  Lesson 
module  completion  times  and  scores  were  not  different,  PV  and  AG  condition  achievement  scores  did  not 
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Table  35.  Attitude  Scale  Alpha 
Reliability  Coefficients 


Seal* 

a 

N 

Lesson  Effectiveness 

.73 

78 

Gaming  Affect 

.86 

78 

TO  Subject  Matter  Favorability 

.70 

78 

Competition 

.23 

78 

Table  36.  Condition  Means  for  Attitude  Instrument  Scales 


Scale 

Non*PLATO 

Condition 

PLATO  Version 

Adjunct  Gaming 

M 

N 

M 

N 

M 

N 

Lesson  effectiveness 

5.16 

31 

5.76 

23 

5.0 

24 

Gaming  affect 

5.15 

31 

6.43 

23 

6.12 

24 

TO  affect 

3.49 

31 

5.11 

23 

3.70 

24 

Table  37.  Analysis  of  Variance: 
Technical  Order  Affect  Scale  Scores 


Source 

DF 

ss 

MS 

F 

P 

Condition 

2 

38.38 

19.19 

6.8 

<01 

Within  Groups 

75 

211.41 

2.82 

differ  significantly,  and  attitude  differences  were  for  the  most  part  unimportant.  The  one  significant  lesson 
completion  time  difference  was  probably  spurious,  and  could  not  have  been  related  to  gaming  since  it 
occurred  in  the  first  lesson  module  which  included  no  game.  Tire  substantive  impacts  were  not  favorable. 
Efficiency  was  reduced  and  attitudes  toward  the  subject  area  were  much  lower.  It  was  evident  that  this 
application  of  games  had  no  specific  or  general  facilitative  effect  on  the  acquisition  of  information. 

It  would  be  a mistake  to  try  and  generalize  these  findings  past  this  immediate  environment.  Though 
the  games  were  characteristically  dissimilar,  tire  training  strategy  was  the  same  in  each;  i.e.,  the  students 
were  asked  questions  in  the  hope  that  they  would  be  stimulated  by  this  activity.  Many  other  gaming 
strategies  with  methods  that  required  the  student  to  process  or  use  information  were  possible.  Thus, 
instructional  design  personnel  might  be  wary  of  games  which  merely  ask  the  student  questions,  but  this 
feeling  should  not  extend  to,  say,  competitive  simulations  which  require  the  student  to  perform  job  tasks  in 
a gaming  atmosphere. 

Both  the  internal  and  external  validity  of  this  study  were  questionable.  Performance  data  collection 
was  suspect  owing  to  “bugs"  in  the  data  collection  routine.  Obviously  poor  data  were  not  included  within 
the  analysis;  however,  the  validity  of  the  remainder  are  suspect.  AG-condition  attitudes  data  may  reflect 
sequence  router  “bugs”  which  sent  students  to  the  wrong  lessons.  Game  time  variability  was  very  high, 
since  specifications  to  limit  interaction  times  were  not  followed.  Numerous  other  problems  could  be 
explicated,  but  the  effect  is  clear:  the  findings  are  contaminated  and  questionable. 
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Another  observation  which  was  consistent  with  other  evaluation  results  should  be  mentioned.  The 
PLATO  version  of  the  TO  lesson  was  superior  to  the  lock-step  instructor-presented  lesson.  It  was  more 
efficient  (5  hours  as  opposed  to  6 hours)  and  produced  more  favorable  student  attitudes  toward  the  subject 
matter. 


X.  OVERALL  CONCLUSIONS  AND  RECOMMENDATIONS 

Overall  Conclusions 

The  following  is  a listing  of  the  overall  service  test  conclusions  which  establish  a basis  for  some  of  the 
final  recommendations. 

1.  PLATO,  as  an  instructional  medium  for  technical  training,  was  instructionally  effective  for 
presenting  task-related  cognitive  materials. 

2.  The  Chanute  PLATO  application  was  not  cost-effective  when  compared  to  courses  employing  less 
sophisticated  individualized  media  to  perform  similar  functions.  The  potential  for  a cost-effective 
application  exists  through  (a)  deriving  cost-effective  CA1  applications,  (b)  deriving  additional  PLATO  CMI 
roles,  (c)  attempting  alternative  course  organizational  structures  to  fully  exploit  computer  applications  and 
minimize  instructor  and  administrative  staff  requirements,  and  (d)  maximizing  terminal  usage. 

3.  Several  conclusions  resulted  from  examining  PLATO’s  impact  on  some  important  aspects  of 
training.  From  the  standpoint  of  student  and  instructor  attitudes,  PLATO  was  an  acceptable  and  even 
desirable  facet  of  the  training  environment.  PLATO’s  incorporation  within  technical  training  was 
compatible  with  existing  organizational  resistance  to  the  introduction  of  PLATO;  resistance  was  considered 
to  be  a nor/nal  and  expected  reaction  to  a major  innovation.  Little  in  the  way  of  new  training  capabilities 
was  demonstrated. 

4.  Courseware  preparation  was  a complex  process  and  more  time-consuming  than  for  less 
sophisticated  media.  The  feasibility  of  the  single  instructor/author  concept  in  the  preparation  of 
cost-effective  computer  applications  was  found  to  be  questionable.  Greater  efficiency  in  materials 
preparation  resulted  from  a team  approach  employing  role  specialists. 

5.  In  terms  of  potential  applications,  the  Chanute  effort  was  limited  owing  to  constraints  in 
management,  resource  acquisition,  personnel  training  and  experience.  Plato  resource  instability  was  derived 
from  its  evolving,  experimentally  oriented  nature,  and  traditional  views  toward  applying  an  innovation. 

6.  Some  training  management  difficulties  were  experienced  because  PLATO  was  not  an  Air  Force 
resource.  Adjustments  were  necessary  in  the  areas  of  training  activity  scheduling  and  course  instructional 
design  selection. 

7.  Few  usage  problems  were  present  in  terms  of  student  interaction  or  courseware  authoring. 
PLATO  was  found  to  be  a reliable  tool,  once  system  expansion  and  experimentation  were  moderated. 

Recommendations 

The  following  recommendations  were  prepared  after  considering  the  total  experience  with  the 
PLATO  IV  service  Test.  They  originate  from  the  empirical  and  qualitative  observations  made  during  the 
incorporation  of  PLATO  IV  into  the  military  technical  training  environment.  The  recommendations  are 
grouped  in  three  categories:  (a)  those  which  are  specific  to  Chanute,  (b)  those  related  to  the  development 
of  CAI  materials,  and  (c)  those  concerned  with  PLATO  applications. 

Recommendations  Specific  to  Chanute 

I.  If  PLATO  is  continued  as  an  operational  medium,  the  Technical  School  Operations  Division 
should  encourage  and  guide  future  PLATO  applications  by  establishing  a functional  area  within  the 
Training  Operations  Division  to  mai  age  expanded  usage  of  PLATO.  To  attain  cost  effectiveness,  a PLATO 
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applications  section  should  be  created  within  the  Training  Services  Division.  This  organization  would  be  the 
focal  point  for  current  and  future  PLATO  applications  and  should  be  provided  with  adequate  resources; 
e.g.,  trained  personnel.  The  consolidation  of  personnel  resources  will  provide  necessary  organizational 
importance,  allow  for  improved  courseware  production,  as  well  as  prevent  deterioration  of  authoring  skills. 
PLATO  applications  should  be  selectively  expanded  to  other  corse  areas,  wherein  CAl  capabilities  might  be 
used  to  solve  training  problems,  increase  instructional  efficiency,  reduce  training  resource,  and 
requirements.  Finally,  a detailed  long-range  plan  should  be  developed  to  identify  organizational  structure, 
manpower  needs,  objectives  and  guidelines  for  systematic  expansions,  internal  program  evaluation,  and  the 
numerous  other  elements  necessary  to  achieve  a well-defined  program. 

2.  If  the  present  application  is  continued,  the  Special  Purpose  Vehicle  Repairman  course  should  be 
self-paced.  This  would  take  advantage  of  CAI’s  individualized  nature,  limit  the  need  for  enrichment 
materials,  and  provide  for  more  efficient  use  of  available  terminals. 

3.  Terminal  utilization  within  the  SPV  course  should  be  maximized  by  determining  the  fewest 
number  of  terminals  required  to  maintain  a given  approach.  Additional  terminals  should  be  shifted  to  other 
courses  so  that  other  cost-effective  applications  of  CAl  may  be  exploited. 

4.  Direct  communication  between  authors  and  instructors  should  be  promoted  to  facilitate 
application  of  PLATO,  and  increase  instructional  efficiency. 

5.  Indirect  lighting  should  be  installed  in  the  PLATO  terminal  room  to  eliminate  the  interfering  glare 
from  the  plasma  panels. 

6.  The  number  of  instructors  in  the  self-paced  course  version  should  be  reduced.  An  instructor  aide 
concept,  where  noninst ruction al  activities  associated  with  student  management  are  performed  by  a less 
expensive  resource,  should  be  implemented. 

Development  of  CAl  Materials 

1.  The  PLATO  microfiche  procurement  process  should  be  analyzed,  then  improved,  for  greater 
efficiency. 

2.  Redesign  of  the  microfiche  projector  is  desirable  to  eliminate  the  focus  and  registration  problems. 

3.  CAl  courseware  should  be  developed  through  a team  approach,  using  specialists  who  have 
overlapping  knowledge  of  other  team  member’s  roles  in  the  lesson  development  process. 

4.  A training  program  should  be  provided  for  individuals  who  perform  courseware  development 
functions.  This  program  would  emphasize  instructional  programming  concepts  used  for  preparing 
individualized  training  materials. 

5.  Instructor  and  supervisor  involvement  should  be  increased  in  the  review  and  revision  of  PLATO 
courseware. 

PLATO  Applications 

1.  To  minimize  the  effect  of  traditional  attitudes  toward  computer  applications,  potentially 
cost-effective  CAl  and  CM1  applications  should  be  derived.  At  Chanute,  PLATO  could  be  used  for; 

a.  Performance  measurement  in  task-related  training  objectives; 

b.  Data  analysis,  to  automatically  provide  more  useful  test  information,  such  as  reliability, 
validity,  item  difficulties,  rather  than  the  descriptive  information  currently  gathered  in  block  measurement 
programs  through  time-consuming  methods; 

c.  Simulation  of  complex  equipment  and/or  tasks; 

d.  Managing  student  progress; 

e.  Assuming  the  interrogative  role  of  instructors  to  monitor  learning  from  sources  other  than 

PLATO; 
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f.  Teaching  direct,  task-related  skills  and  procedural  knowledges  rather  than  simple  principles 
and  fundamentals;  and 

g.  Continuous  monitoring  of  training  effectiveness;  i.c.,  collection  and  analysis  of  data 
concerning  training  content,  training  time,  learning  achievement,  and  student  abilities/aptitudes  for  each 
training  objective. 

2.  Criteria  should  be  established  for  selecting  potentially  cost-effective  applications  and/or  those 
applications  which  substantially  improve  methods,  but  on  which  no  cost  value  can  be  placed.  Some 
candidate  applications  are: 

a.  Instructional  modules  where  low  efficiency  with  existing  media  is  present; 

b.  Areas  where  reductions  could  be  made  in  expensive  resources  (instructors,  equipment); 

C.  Training  tasks  where  increased  safety  is  required;  and 

d.  Training  sequences  where  greater  productivity/less  mistakes  could  be  achieved  by  continuous 
monitoring  of  student  performance. 

3.  It  is  very  important  that  findings  of  other  CAI  projects  ue  reviewed  to  incorporate  those  results 
which  are  applicable  to  training  environments. 

4.  Different  CAI  strategies  should  be  compared  to  determine  if  greater  efficiency  can  be  obtained 
for  different  subject  matter  areas  and  student  abilities. 

Final  Remarks 

Computer-based  instruction  will  be  expanded  throughout  Air  Force  technical  training  because  of  the 
desire  to  use  technology  to  improve  the  training  environment.  Testing  PLATO  within  the  Special  Purpose 
Vehicle  Repairman  courses  assisted  in  this  endeavor.  The  evaluation  findings  demonstrated  that  PLATO,  as 
an  instructional  medium,  could  be  successfully  incorporated  into  the  technical  training  process.  Its  training 
effectiveness,  impact  on  attitudes,  reliability  and  utility  were  acceptable.  The  only  major  disadvantage  was 
that  the  cost  of  instruction  was  higher  than  it  would  be  with  less  sophisticated  media  used  for  the  same 
applications.  Yet,  it  is  a mistake  to  think  of  PLATO  as  just  as  instructional  medium.  It  should  be  considered 
as  an  educational  tool,  with  potentially  cost-effective  applications  in  the  areas  of  administration  and 
student  management,  as  well  as  training  instruction.  When  it  is  understood  how  to  best  use  this  tool,  cost 
effectiveness  can  be  attained. 
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APPENDIX  A:  SPECIAL  PURPOSE  VEHICLE  JOB  TASKS 


Practice  Automotive  Personnel  and  Equipment  Shop  Safety 

1 . Apply  safety  practices  with  tools  and  equipment 

2.  Practice  housekeeping 

Vehicle  Maintenance  Publications 

3.  Use  technical  orders 

Maintenance  and  Inspection  System  and  Forms 

4.  Use  maintenance  data  collection  forms 

5.  Use  man-hour  accounting  forms 

Use  Maintenance  Tools  and  Equipment 

6.  Hand  tools 

7.  Special  tools 

8.  Test  equipment 

Engine  Maintenance 

9.  Disassemble  and  reassemble  engines 

10.  Inspect  engine  parts  for  defects  and  wear  Test,  troubleshoot,  service,  and/or  adjust  engine; 

11.  Valve  trains 

12.  Lubrication  systems 

13.  Cooling  Systems 

14.  Fuel  Systems  (gas) 

1 5.  Fuel  Systems  (diesel) 

16.  Air  and  exhaust  system 

17.  Emission  control  and  anti-pollution  devices 

Electrical  Systems  Maintenance 

1 8.  Use  electrical  schematics  and  diagrams  Test,  troubleshoot,  and  maintain: 

19.  Starting  systems 

20.  Charging  systems 

21.  Ignition  systems 

22.  Lighting  systems 

23.  Warning  systems 

24.  Batteries 

Hydraulic  System  Maintenance 

25.  Use  hydraulic  schematics  and  diagrams 
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26.  inspect  and  troubleshoot  hydraulic  systems 

27.  Service,  adjust,  and  test  hydraulic  system  components 

Power  Train  Maintenance 

28.  Disassemble,  reassemble,  service,  and  adjust  power  train  systems  and  components 

29.  Troubleshoot  and  test  power  train  system 

Brake  Systems  Maintenance 

30.  Disassemble,  reassemble,  service,  adjust  and  test  brake  systems 

Mechanical  and  Power  Streering  Maintenance 

Disassemble,  reassemble,  service,  test,  and  adjust: 

3 1 . Mechanical  steering  system 

32.  Power  steering  system 

Heater  Maintenance 

33.  Inspect,  service,  troubleshoot,  adjust  and  test  heating  system 


BLOCK  # 

SHRED  A B C -1 

PLATO  QUESTIONNAIRE 

NAME SSAN DATE 

Circle  the  number  that  represents  how  you  feel  about  the  following 
statements. 


Very  Very 

Strongly  Strongly 
Disagree Agree 

1.  I enjoyed  the  time  I spent  on  PLATO.  01  23456789 

2.  I learned  what  the  PLATO  lessons  tried  to  teach.  01  23456789 


3.  I would  like  to  study  more  PLATO  lessons. 

4.  I feel  PLATO  didn't  teach  me  very  much. 

5.  I feel  challenged  to  do  my  best  work  on  PLATO. 

6.  What  have  you  enjoyed  most  about  using  PLATO? 
(You  may  use  the  other  side). 


0123456789 
0123456789 
01  23456789 


7.  If  you  could  change  anything  about  PLATO  what  would  you  change? 
(You  may  use  the  other  side). 
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EVALUATION  OF  PLATO  IV  IN  VEHICLE  MAINTENANCE  TRAINING. (U) 
NOV  77  B E OALLMAN'  P J DELEO*  P S MAIN 


UNCLASSIFIED  AFhRL-TR-77-59  NL 


BLOCK  # 

SHRED  A B C -1 


BLOCK  QUESTIONNAIRE 


Circle  the  number  that  represents  how  you  feel  about  the  following 
statements. 


1.  I enjoyed  the  time  I spent  in  class. 


Very  Very 

Strongly  Strongly 

Disaqree  Aqree 

0 1 2 3 4 5 6 7 8 9 


2.  I learned  what  the  instructor  and  the  material  01  23456789 
tried  to  teach. 

3.  I would  like  to  study  more  with  this  instructor.  01  23456789 

4.  I feel  the  instructor  and  the  material  didn't  01  23456789 

teach  me  very  much. 

5.  I feel  challenged  to  do  my  best  work  in  this  0123456789 
class. 

6.  What  have  you  enjoyed  most  about  this  class? 

(You  may  use  the  other  side). 


7.  If  you  could  change  anything  about  this  class  what 
would  you  change?  (You  may  use  the  other  side). 
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SSAN 


PLATO  QUESTIONNAIRE 
SECTION  II 


1.  The  method  by  which  I was  told  whether  I had 
given  a right  or  wrong  answer  became  monotonous. 

2.  Nobody  really  cared  whether  I learned  the  course 
material  or  not. 

3.  I felt  challenged  to  do  my  best  work. 

4.  I felt  isolated  and  alone. 

5.  I felt  as  if  someone  were  engaged  in  conversation 
with  me. 

6.  As  a result  of  having  studied  by  this  method,  I am 
interested  in  learning  more  about  the  subject 
matter. 

7.  I was  more  involved  in  operating  the  terminal  than 
in  understanding  the  course  material. 

8.  The  learning  was  too  mechanical. 

9.  I felt  as  if  I had  a private  tutor. 

10.  The  equipment  made  it  difficult  to  concentrate  on 
the  course  material. 

11.  The  situation  made  me  quite  tense. 

12.  PLATO  as  used  in  this  course,  is  an  inefficient 
use  of  the  student's  time. 

13.  My  feeling  toward  the  course  material  after  I had 
completed  the  PLATO  portion  of  the  course  was 
favorable. 

14.  I felt  frustrated  by  the  situation. 

15.  ’ found  the  computer-assisted  instruction  approach 
in  this  course  to  be  inflexible. 

16.  Material  which  is  otherwise  interesting  can  be 
boring  when  presented  on  PLATO. 


Very  Very 

Strongly  Strongly 

Disagree Agree 
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17.  I was  satisfied  with  what  I learned  while 
taking  the  course. 

18.  In  view  of  the  amount  I learned,  this  method 
seems  superior  to  classroom  instruction  for 
many  courses. 

19.  I would  prefer  PLATO  to  traditional  instruction. 

20.  PLATO  instruction  is  just  another  step  toward 
de-personal i zed  instruction. 

21.  I was  concerned  that  I might  not  be  under- 
standing the  material. 

22.  The  responses  to  my  answers  seemed  appropriate. 

23.  I felt  uncertain  as  to  my  performance  in  the 
programed  course  relative  to  the  performance 
of  others. 

24.  I was  not  concerned  when  I missed  a question 
because  nobody  was  watching  me. 

25.  I found  myself  just  trying  to  get  through  the 
material  rather  than  trying  to  learn. 

26.  I knew  whether  my  answer  was  right  or  wrong 
before  I was  told. 

27.  In  a situation  where  I am  trying  to  learn  some- 
thing, it  is  important  to  me  to  know  where  I 
stand  relative  to  others. 

28.  I guessed  at  the  answers  to  some  questions. 

29.  I was  aware  of  efforts  to  suit  the  material 
specifically  to  me. 

30.  I was  encouraged  by  the  responses  given  to  ray 
answers  of  questions. 

31.  In  view  of  the  time  allowed  for  learning,  I felt 
too  much  material  was  presented. 


Very 

Strongly 

Disagree 

0 12  3 4 

0 12  3 4 

0 12  3 4 
0 12  3 4 

0 12  3 4 

0 12  3 4 
0 12  3 4 

0 12  3 4 
0 12  3 4 
0 12  3 4 
0 12  3 4 

0 12  3 4 
0 12  3 4 

0 12  3 4 

0 12  3 4 


Very 

Strongly 

Agree 

5 6 7 8 9 

5 6 7 8 9 

5 6 7 8 9 
5 6 7 8 9 

5 6 7 8 9 

5 6 7 8 9 
5 6 7 8 9 

5 6 7 8 9 
5 6 7 8 9 
5 6 7 8 9 
5 6 7 8 9 

5 6 7 8 9 
5 6 7 8 9 

5 6 7 8 9 

5 6 7 8 9 
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Very  Very 

Strongly  Strongly 

Di sagree  Agree 


32.  I entered  wrong  answers  in  order  to  get  more 
information  from  the  machine. 

33.  I felt  I could  work  at  my  own  pace. 

34.  Questions  were  asked  which  I felt  were  not 
related  to  the  material  presented. 

35.  Material  which  is  otherwise  boring  can  be 
interesting  when  presented  by  PLATO. 

36.  I could  have  learned  more  if  I hadn't  felt 
pushed. 

37.  I was  given  answers  but  still  did  not  under- 
stand the  questions. 

38.  The  course  material  was  presented  too  slowly. 

39.  The  responses  to  my  answers  seemed  to  take  into 
account  the  difficulty  of  the  question. 

40.  While  on  PLATO,  I encountered  mechanical 
malfunctions. 

41.  Computer-assisted  instruction  did  not  make  it 
possible  for  me  to  learn  quickly. 

42.  PLATO  could  be  much  better  if  the  lessons 
were  improved. 

43.  The  lessons  on  PLATO  were  interesting  and  really 
kept  me  involved. 

44.  What  I learned  from  PLATO  made  the  classroom  and 
laboratory  instruction  easier  to  understand. 

45.  The  PLATO  lessons  were  dull  and  difficult  to 
follow. 
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PLATO  QUESTIONNAIRE 
SECTION  III 


As  a student,  I do  my  best  work  with 


1.  Movies  and  filmstrips. 

2.  PLATO  lessons. 

3.  Study  guides  and  workbooks. 

4.  Instructor  lectures. 

5.  Laboratory  instruction. 

6.  Porgrammed  texts. 


Working  with  the  PLATO  system: 

7.  Is  fun. 


8.  Is  frustrating. 

9.  Is  challenging. 

10.  Is  annoying. 

11.  Is  confusing. 

12.  Makes  me  proud  of  myself. 

13.  Is  boring. 

14.  Is  relaxing. 

15.  Is  depressing. 

16.  Is  enjoyable. 

17.  Is  de-personalizing. 

18.  Is  exciting. 


Very  Very 

Strongly  Strongly 

Di saaree  Agree 
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100 


io% 

of  the 
Time 


100% 
of  the 
Time 


19.  How  often  has  PLATO  worked  when  you  have  01  23456789 

attempted  to  use  it? 

20.  During  how  many  sessions  have  the  mechanical  01  23456789 
interruptions  made  you  want  to  stop  using 

PLATO? 

21.  During  your  school  day,  if  you  had  your  choice,  01  23456789 
how  much  of  your  time  would  you  spend  working 

with  PLATO? 


APPENDIX  D:  RESULTS  OF  LONG  FORM  FACTOR  ANALYSIS 


Item  # 


3 

5 

6 
9 

13 

17 

18 
19 
22 
30 
35 
41 

43 

44 
47 
47 
52 
54 

57 

58 

59 
61 

63 

64 
66 


14 

53 

55 

56 
60 


11 

23 

31 

33 

34 

36 

37 


48 

49 
51 


2 

4 

8 

10 


Description 


Factor  l - PLATO  Instruction 

I felt  challenged  to  do  well 

Felt  as  if  in  conversation 

More  interest  as  result  of  study 

Felt  I had  private  tutor 

Feelings  toward  course  material  favorable 

Satisfied  with  info  learned 

PLATO  method  superior  relative  to  amount  learned 
Prefer  PLATO  to  traditional  instruction 
Responses  to  answers  seemed  appropriate 
Encouraged  by  responses  to  my  answers 
Boring  material  can  be  interesting  on  PLATO 
CAI  did  not  make  learning  quicker 
Lessons  on  PLATO  interesting  and  kept  me  involved 
PLATO  instruction  helped  in  lab  and  classroom 
PLATO  lessons  were  dull  and  difficult  to  follow 
Do  best  work  with  PLATO  lessons 
PLATO  is  fun 
PLATO  is  challenging 

Makes  me  proud  of  myself  (accomplishment) 

PLATO  is  boring 
Is  relaxing 
Is  enjoyable 
Is  exciting 
Reliability 

Amount  of  time  desired  on  PLATO 

Factor  II  - Frustration 

1 felt  frustrated  by  situation 
PLATO  is  frustrating 
PLATO  is  annoying 
PLATO  is  confusing 
PLATO  is  depressing 

Factor  111  - Training  Stress 

The  situation  made  me  tense 
I felt  uncertain  about  my  performance 
I felt  too  much  material  was  presented 
I felt  I could  work  at  my  own  pace 
Irrelevant  questions  were  asked 
I felt  pushed 

I got  answers  but  still  did  not  understand  questions 
Factor  IV  — Non-PLATO  Media 

I do  best  work  with  study  guide  workbooks 
l do  best  work  with  instructor  lectures 
I do  best  work  with  programmed  texts 

Factor  V - Mechanistic  and  Impersonal  Leaning 

No  one  cared  if  I learned  or  not 
I felt  isolated  and  alone 
The  learning  was  too  mechanical 
The  equipment  interfered  with  learning 


Factor  Loading 


.764 

.695 

.788 

.691 

.762 

.733 

.767 

.768 

.508 

.687 

.717 

.506 

.737 

.794 

.624 

.752 

.844 

.750 

.714 

.633 

.531 

.747 

.721 

.518 

.724 


.649 

.735 

.769 

.707 

.575 


.482 

.447 

.768 

.450 

.417 

.608 

.440 


.863 

.673 

.833 


.501 

.493 

.657 

.526 
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SECTION  I: 


TECHNICAL  TRAINING  STUDENT  SURVEY 
PE  7403 

AIR  FORCE  HUMAN  RESOURCES  LABORATORY 


GENERAL  INSTRUCTIONS: 

1.  The  items  contained  on  this  form  are  designed  to  measure  student 
attitudes  toward  Air  Force  technical  training. 

2.  The  form  is  intended  to  give  you  the  opportunity  to  help 
improve  student  training. 

3.  It  is  very  important  that  your  answers  reflect  your  true  feelings. 
This  is  not  a test  and  you  are  not  required  to  put  your  name  on 
the  form. 

4.  Please  carefully  follow  the  instructions  at  the  beginning  of 
each  of  the  four  main  sections  of  this  form. 


OOOOOOOOOOOO 

zo)5<cJzjSo.h>y 

<iu5IL53D^ujyoiu 
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1.  Below  are  statements  describing  rewards  a student  might  receive  if  he  performs  well  in  technical  traininc 

2.  Beside  each  statement  are  two  separate  rating  scales. 

On  Scale  1 indicate  how  likely it  is  for  you  to  receive  the  rewaro  if  you  perform  well  in  training. 


On  Scale  '2  indicate  how  important  the  reward  is  *o  you.  Consider  only  its  importance,  not  how  likely  or  unlikely  you  are  to 
receive  the  reward. 

3.  Notice  that  each  scale  has  five  circles.  The  words  above  the  scales  describe  the  meaning  of  the  circles  at  the  ends  of  each  scale.  The 
three  circles  in  the  middle  of  each  scale  represent  feelings  between  those  described  at  the  scale  ends.  You  might  want  to  think  of 
each  scale  as  similar  to  a thermometer  lying  on  its  side. 

4.  Answer  each  item  by  darkening  one  circle  on  each  scale  to  indicate  how  you  feel  about  the  statement.  Read  each  statement  carefully 
and  take  all  the  time  you  need. 


SCALE  1: 

IF  YOU 

PERFORM  WELL 


SCALE  2: 

HOW  IMPORTANT 
TO  YOU 


1 . Increased  iob  security  after  graduating  from 


Very 

Unlikely 


Very  Not 
Likely  Important 


Very 

Important 


technical  school 


2.  Faster  promotion 

3.  Greater  chance  to  participate  in  important  de- 
cisions after  graduating  from  technical  school  . 

4.  More  challenging  duty  assignments  after 

graduating  from  technical  school 

5.  More  job  responsibilities  after  graduating  from 

technical  school 

6.  Greater  chance  of  being  skilled  and  competent 

in  your  career  field 

7.  Increased  chance  of  getting  a good  civilian 

job  after  Air  Force  service 

8.  Greater  chance  to  be  assigned  to  your  base 

of  choice . 

9.  Increased  off-duty  privileges  (for  example, 

three-day  passes  or  no  squadron  detail) 

10.  Greater  freedom  in  deciding  how  to 

accomplish  class  work 

11.  I ncreased  chance  of  being  admired  and 

respected  by  fellow  students 


CZD  CTD 


ATC  Form  1631,  Jun  74 


GO  TO  THE  NEXT  PAGE 


SECTION  I: 


12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 

21. 

22. 

23. 

24. 

25. 

26. 


Instructors  pay  more  attention  to  your 

ideas  and  suggestions 

Increased  educational  growth  and 

development 

Greater  chance  to  help  other  students  learn 

the  subject  matter 

Greater  chance  to  do  better  on  tests  and 

receive  better  grades 

Receive  compliments,  recognition  and  praise 

from  instructors 

Fewer  "Mickey  Mouse"  duties  in  the 

Squadron  

Fewer  "Mickey  Mouse"  assignments  in 

class  

Feeling  of  set!  respect  and  sense  of 

accomplishment 

Increased  opportunity  to  use  your 

abilities 

Receive  more  challenging  class 

assignments 

Greater  opportunity  to  study  subject  mat- 
ter of  special  interest  to  you 

Increased  chance  of  completing  training 
ahead  of  schedule 

Provided  with  more  spare  time 

Instructors  less  critical  of  your  work 

Increased  chance  of  being  an  "Honor" 
graduate 


SCALE  1. 

IF  YOU 

PERFORM  WELL 


SCALE  2: 

HOW  IMPORTANT 
TO  YOU 
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SECTION  II: 

1.  Please  use  the  scales  below  to  describe  your  SSAN  of  main  (lead)  instructor. 

2.  Darken  the  one  circle  on  each  scale  that  best  expresses  your  feelings. 


27. 

Ineffective 

CO  CO 

1 1 

CZD 

1 

Effective 
CO  CO 

1 1 

34.  Unprepared 
CO  CO 

1 1 

CO 

1 

CO 

1 

Prepared 

CD 

41.  Considerate 
CD  CD 

i 1 

CD 

1 

Inconsiderati 
CD  CD 

i l 

28 

Knowledgeable 

CO  CO 

1 1 

CD 

i 

Ignorant 
CD  CD 

i 1 

35.  Intelligent 

CD  CD 

i i 

CD 

1 

CD 

1 

Stupid 

CD 

42.  Hinders 

CD  CD 

i i 

CD 

i 

Helps 
CD  CD 

i i 

29 

Boring 

CO  CO 

I 1 

CO 

1 

Interesting 

CO  CO 

1 1 

36.  Inefficient 

CO  CO 

1 1 

CO 

1 

CD 

1 

Efficient 

CD 

43.  Friendly 

CO  CO 

1 1 

CO 

1 

Unfriendh 
CD  CD 

1 1 

30. 

Dependable 

CD  (D 

L 1 

CD 

i 

Undependable 
CD  CD 

i i 

37.  Encourages 

CD  CD 

( i 

CD 

1 

Discourages 
CD  CD 

i l 

44.  Supportive 
CD  CD 

■ i 

CO 

Hostile 
CD  CD 

■ i 

31. 

Disorganized 

CO  CO 

1 1 

CO 

1 

Organized 

CO  CO 

1 1 

38.  Criticizes 
(D  CD 

t i 

CD 

1 

CO 

1 

Praises 

CD 

45.  Ridicules 

CD  CD 

1 i 

CD 

Compliments 
CD  CD 

i l 

32. 

Unsure 

CO  CO 

1 I 

CO 

1 

Confident 
CO  CO 

1 1 

39.  Fair 

CO  CO 

1 1 

CZD 

CD 

i 

Unfair 

CD 

46.  Cooperative 
CD  CD 

1 l 

CD 

i 

Uncooperative 
CD  CO 

l l 

33. 

Convincing 

CO  CO 

1 1 

CO 

1 

Unconvincing 
CO  CO 

1 1 

40.  Impatient 

CO  CO 

1 1 

CO 

1 

CO 

1 

Patient 

CD 
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SECTION  III: 

1 . Below  are  a series  of  statements  related  to  both  your  training  and  training  environment. 

2.  Please  darken  the  one  circle  on  each  scale  that  best  expresses  your  feelings. 


Definitely 

Disagree 


47.  Certain  students  are  hostile  toward  other  class  members 

48.  Most  students  get  along  well  together 

49.  Fellow  students  look  out  for  each  other 

50.  Certain  students  are  uncooperative 

51 . Certain  students  are  responsible  for  petty  quarrels  and  bad  feelings  among  class  members  .... 

52.  There  are  tensions  among  some  students  which  interfere  with  training  activities 

53.  Certain  students  are  incapable  of  working  together 

54.  Students  help  each  other  to  learn  the  necessary  course  material 

55.  Some  students  are  not  liked  or  accepted  by  fellow  students 

56.  Students  have  to  take  advantage  of  others  in  order  to  succeed  in  training 

57.  Students  are  given  an  equal  opportunity  to  demonstrate  their  capabilities 

58.  Students  are  subject  to  strict  discipline 

59.  Student  training  is  too  closely  supervised 

60.  Students  are  encouraged  to  speak  their  minds  even  if  it  means  disagreeing  with  the  instructors  . 

61.  Students  are  encouraged  to  suggest  improvements  or  solutions  to  training  problems 

62.  Students  are  encouraged  to  participate  in  classroom  discussions 

63.  Students  are  given  the  opportunity  to  participate  in  class 

64.  Student  suggestions  and  recommendations  are  considered  with  fairness 

65.  Students  are  seldom  able  to  use  their  own  judgment 

66.  Students  have  no  say  about  what  happens  to  them 

67.  Students  have  little  chance  to  influence  the  way  the  class  is  conducted 

68.  Students  have  the  freedom  to  establish  their  own  study  schedules 

69.  Spare  time  in  class  may  be  spent  as  each  student  sees  fit 

70.  Students  are  rarely  given  the  chance  to  freely  express  their  ideas  in  the  classroom 
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Definitely 

Disagree 


Definitely 

Agree 


71 . Students  are  seldom  allowed  to  act  independently 

72.  Pressure  for  perfection  is  unbearable 

73.  The  military  atmosphere  in  the  classroom  interferes  with  learning  of  the  subject  matter 

74.  Squadron  duties  interfere  with  study 

75.  In  order  to  do  well  in  training,  students  have  to  do  things  that  are  against  their  personal  values. 

76.  Students  don't  know  what  is  expected  of  them 

77.  There  is  confusion  in  the  planning  and  organi2ation  of  classroom  activities 

78.  There  is  considerable  conflict  among  training  objectives 

79.  Performance  standards  are  unreasonably  high 

80.  Emphasis  is  placed  on  passing  the  course  rather  than  learning  subject  matter 

81.  There  is  a good  deal  of  disagreement  on  how  this  training  should  be  conducted 

82.  The  student  workload  is  too  heavy 

83.  The  quantity  of  class  work  interferes  with  how  well  it  is  done 

84.  Emphasis  on  military  bearing  and  appearance  detract  from  student  performance 

85.  Training  hours  are  too  long 

86.  Conflicts  exist  in  the  training  requirements 

87.  Training  equipment  (including  trainers)  is  adequate 

88.  Training  equipment  (including  trainers)  is  readily  available  for  student  practice 

89.  Time  allowed  on  training  equipment  (including  trainers)  is  sufficient 

90.  Training  evaluation  or  testing  is  an  accurate  indication  of  student  performance 

91 . Study  guides  are  difficult  to  understand 

92.  Excessive  attention  is  given  to  unimportant  details 

93.  Course  materials  are  so  poor  that  they  contribute  little  to  learning 

94.  Course  materials  are  not  closely  related  to  the  course  objectives 

95.  Course  materials  are  more  difficult  than  they  should  be 

96.  My  progress  in  class  is  not  what  it  should  be  due  to  the  poor  quality  of  training  or  course 

materials 

97.  Classroom  temperature  is  satisfactory 

98.  Dormitory  sleeping  facilities  are  adequate 
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SECTION  III: 


(S>0<S0<3>C5K®CE)<®<2> 

<3>©(*>®<3>(3><3><S>(3><S> 

<3>  © ® ® <3>  (5>  <3>  © CS>  C5> 

(§)©©©(5>(s>(jD<S><3>®> 

<§>  © ® ® <S>  <3>  ($  <5>  CS>  <3> 

<S>  © ® 3 <2>  <S)  <3>  ® <3>  <*> 

<5>  © (3  <3  <3>  <3>  <3>  <S>  CS>  <3> 

99.  Classroom  lighting  is  adequate 

100.  Classroom  chairs  art  comfortable  

101.  Classroom  seating  arrangement  is  satisfactory  . . 

102.  Length  of  class  breaks  is  about  right 

103.  Number  of  class  breaks  is  sufficient 

104.  Study  facilities  outside  the  classroom  are  adequate 

105.  Classroom  ventilation  is  about  right 

106.  Time  allowed  for  testing  is  sufficient 

107.  Classroom  noise  control  is  effective 


106.  Classroom  work  space  is  sufficient  (desk  or  table  top  area) 

109.  Supplementary  study  materials  (manuals,  regulations,  technical  orders,  etc.)  are  readily 
available  for  student  use 


1 10.  Base  recreation  facilities  are  adequate 

111.  Off  duty  study  time  is  sufficient 

112.  T im«  allowed  for  review  of  tests  is  adequate 

113.  Enough  training  time  is  spent  on  difficult  and  important  subject  matter . 
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1 14.  Some  students  would  perform  better  on  a different  shift 

SECTION  IV:  ' 1 ' 1 

1 Below  are  statements  (bout  your  satisfaction  with  your  training  and  career  field. 

2 Please  darken  the  circle  that  beat  expresses  your  feelings  about  the  statement  in  the  same  way  you  have  in  the  other  sections  of  this  form. 


Completely 

Dissatisfied 


Completely 

Satisfied 


»15 

How  do  you  feel  about  your  technical  training? 

CO 

CO 

CO 

CO 

1 

CO 

tie. 

How  do  you  feel  about  your  assigned  career  field? 

CO 

CO 

» 

CO 

1 

CO 

1 

J=> 
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How  do  you  feel  about  the  Air  Force? 

CO 

CO 

CO 

CO 

CO 

1 

t 

1 

1 

- 1 
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Completely 

Different 

Identical 

118. 

How  similar  is  your  assigned  career  field  to  your  preferred  career  field? 

. . CHD 

c — > 

< > 

t > 

t } 

1 

i 

119. 

How  accurate  was  the  information  you  received  about  your  career  field  before  entering 

Hitftly 

Inaccurate 

Highly 

Accurate 

technical  training? 

. . . CD 

c=> 

— 1- 

CD 

(HD 

i 

CD 

120. 

What  effect  has  technical  training  had  on  your  feelings  about  your  career  field? 

Strongly 

Negative 

cz> 

CD 

l 

(HD 

i 

o 

Strongly 

Positive 

O 

. i 

Definitely 

No 

Definitely 

Yes 

121. 

If  you  have  the  chance,  will  you  change  to  another  career  field? 

. . CD 

CD 

—I 

CD 

1 

CD 

l_ 

cp 

REMARKS: 


APPENDIX  F:  COMPARISON  OF  STUDENT  MOTIVATION  AND  ATTITUDE  FOR  THE  FIRST 
ADMINISTRATION  OF  THE  TECHNICAL  TRAINING  SURVEY 
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1 


VARIABLE 

SAMPLE' 

OBSERVED 

MEAN 

RANGE  OF  POSSIBLE 
POPULATION  MEANS 
(95%  CONFIDENCE) 

CHANUTE 

POPULATION 

MEAN 

ATC 

POPULATION 

MEAN 

PB 

94.63 

92.14  <u<  97.12 

Instrumentality 

CP 

87.03a,b 

80.71  <M<  93.35 

93.64 

93.50 

NP 

91.85 

83.35  <u<  100.35 

BL 

88.60 

63.07  <M<  94.14 

Valence 

PB 

CP 

NP 

BL 

102.40a,b  99.88  <M<  104.92 

98 . 03a ,b  91.92  <y<  104.14 
98.80  88.78  <u<  108.82 

102.30  97.61  <u<  106.97 

105.02 

105.02 

PB 

387.84  371.92  <u<  403.75 

Valence  of 

CP 

342.24a,b  302.59  <p<  381.90 

396.71 

394.78 

Performance 

NP 

368.50  307.88  <p<  429.12 

BL 

360.29a,b  328.33  <p<  392.25 

Instructor 

Technical 

Competence 

PB 

CP 

NP 

BL 

38.56 

31.46a’b 

41.65 

40.79 

37.34  <\i<  39.78 
27.69  <p<  35.22 
38.92  <p<_  44.38 
38.74  <p<_  42.84 

42.41 

42.68 

PB 

a ,b 

37.92  ’ 

36.70  <vi<  39.15 

Instructor 

CP 

32.64  ’ 

29.19  <v<_  36.08 

40.80 

41.03 

Personal 

NP 

40.75 

37.74  43.76 

Relations 

BL 

37.29 

34.42  <p<_  40.16 

Physical 

Setting 

PB 

CP 

NP 

BL 

60.22 
58.09a,b 
61.45  , 

59.18a,b 

59.34  <u<  61.11 
56.13  <pc  60.05 
58.32  <m<  64.58 
57.52  <y<_  60.85 

63.64 

63.36 

PB 

12.51 

12.16  <p<  12.86 

Global 

CP 

11.30 

10.49  <p<  12.12 

11.34 

11.20 

Training 

NP 

11.90 

10.87  <y<_  12.93 

Satisfaction 

BL 

<M< 
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RANGE  OP  POSSIBLE 

CHANUTE 

ATC 

OBSERVED 

POPULATION  MEANS 

POPULATION 

POPULATION 

VARIABLE 

SAMPLE' 

MEAN 

(95 o CONFIDENCE) 

MEAN 

MEAN 

P3 

13.91a»b 

13.57  <u<  14.26 

Career 

CP 

13.39 

12.52  <u<  14.27 

12.70 

12.66 

Choice 

NP 

13.75b 

12.68  <u<  14.82 

Satisfaction 

BL 

12.10 

11.33  <p<_  12.88 

1. 

pb 

= 152;  N 

cp 

= 33;  Nnp  = 20;  Nbl 

= 38 

2.  N = 2484 


3.  N = 12666 

a.  Observed  mean  differed  from  Chanute  population 

b.  Observed  mean  differed  from  ATC  population 
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RANGE  OF  POSSIBLE 
POPULATION  MEANS 
(95%  CONFIDENCE) 


CHANUTE 

POPULATION 

MEAN 


ATC 

POPULATION 

MEAN 


OBSERVED 

MEAN 


VARIABLE 


SAMPLE 


98.51' 

90.83 

93.00 

92A5 


Instrumentality 


Instructor 

Technical 

Competence 


Instructor 

Personal 

Relations 


Fellow 

Students 


Organ i zat iona 1 
Control 


Training 

Stress 


Physical 

Setting 

PB 

CP 

NP 

BL 

68 . 88a  »b 
65.48 

68 . 85b 
67.88 

67.04  <p<  70.72 

61.97  <p<  68.98 

63.60  <p<  74.10 

64.48  <u<  71.28 
— — 

63.64 

63.36 

PB 

12 ,08a>b 

11.68  <y<  12.49 

Global 

CP 

10.76 

9.95  <p<  11.57 

11.34 

11.20 

Training 

NP 

10.85 

9.38  <p<  12.32 

Satisfaction 

BL 

11.98 

11.12  <u<  12.83 

PB 

13.69a’b 

13.11  <u<  14.27 

Career 

CP 

12.29 

11.22  <p<  13.35 

12.70 

12.66 

Choice 

NP 

11.10a’b 

9.59  <p<  12.61 

Satisfaction 

BL 

13.62 

12.23  <u<  15.01 

1 

1.  V = 158;  Ncp  = 42;  Nnp  = 20;  Nbl  42 

2.  N = 2484 

3.  N = 12666 

a.  Observed  mean  differed  from  Chanute  population 

b.  Observed  mean  differed  from  ATC  population 


14.  Approximately  how  many  hours  have  you  spent  on  the  

PLATO  terminal? 

15.  Approximately  how  many  hours  have  you  spent  in  the 

PLATO  lab  area.  (B-105  Truemper  Hall)?  

16.  I enjoy  working  with  the  PLATO  system.  0123456789 

17.  I am  very  positive  concerning  the  use  of  0123456789 

PLATO  in  Air  Force  technical  training. 
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Very  Very 
Strongly  Strongly 
Disagree  Agree 


18.  I believe  PLATO  is  an  efficient  means  of  0123456789 

teaching  students 

19.  The  students  are  being  taught  what  they  0123456789 

need  to  learn  using  PLATO. 

20.  I feel  challenged  to  do  my  best  work  while  0123456789 
instructing  with  the  PLATO  system. 

21.  PLATO  provides  better  training  when  0123456789 

combined  with  the  ISD  process. 

22.  I believe  more  can  be  taught  with  PLATO  0123456789 

than  with  the  system  of  stand-up  instruction. 

Working  with  the  PLATO  system: 

23.  Is  fun.  0123456789 

24.  Is  frustrating.  0123456789 

25.  Is  challenging  0123456789 

26.  Is  annoying.  0123456789 

27.  Is  confusing.  0123456789 

28.  Makes  me  proud  of  0123456789 
myself . 

29.  Is  boring.  0123456789 

30.  Is  relaxing.  0123456789 

31.  Is  depressing.  0123456789 

32.  Is  enjoyable.  0123456789 

33.  Is  de-personalizing.  0123456789 

34.  Is  exciting.  0123456789 

35.  Personally,  I would  like  to  spend  more  time  0123456789 

operating  the  PLATO  terminal. 

36.  Personally,  I think  it  would  be  a good  idea  0123456789 
to  get  more  involved  with  programming  on 

PLATO. 
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37.  The  lessons  on  PLATO  are  well  written. 

38.  The  lessons  on  PLATO  are  for  the  most 
part  technically  accurate. 

39.  PLATO  instruction  will  always  need  to 
be  supplemented  by  normal  instruction. 

40.  I do  not  feel  my  job  is  threatened  by 
PLATO. 


Very 

Strongly 

Disagree 

0 12  3' 

0 12  3' 

0 12  3' 

0 12  3' 


Very 

Strongly 

Agree 

5 6 7 8 9 

5 6 7 8 9 

5 6 7 8 9 

5 6 7 8 9 
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APPENDIX  I:  LESSON  DEVELOPMENT  SCENARIO 


The  first  items  required  to  develop  a lesson  were  the  criterion  objective,  master  validation  examination 
(MVE),  and  the  teaching  points.  We  will  follow  one  author,  Mr.  Arthur,  through  the  process  of  preparing  a 
lesson.  Let  us  say  that  he  selected  the  objective  concerning  clutches  and  obtained  the  necessary  items  from 
the  1SD  team  (see  Figure  II). 

STS  Element  14A  STS  Code  Level  B 

OBJECTIVE:  Without  reference,  identify  the  relationship  of  basic  facts  and  general  operating  principles  of 
clutches  with  75%  accuracy. 

TEACHING  POINTS: 

1 . Clutch  component  identification 

a.  Driving  Members 

(1)  Turn  with  engine 

(2)  Cast  iron  plates 

(a)  Flywheel 

(b)  Pressure  plate 

b.  Driven  Member 

(1)  Friction  plate  (disc) 

(a)  Types 

1 Wet 

2 Dry 

3 Single  disc 

4 Multiple  disc 

(b)  Application 

1 Splined  hub 

2 Dished 

c.  Pressure  Plate  Assembly 

(1)  Cover 

(2)  Springs 

(a)  Coil  (small) 

(b)  Helical  (large) 

(3)  Diaphragm 

(4)  Release  Levers 

d.  Linkage 

(1)  Mechanical 

(a)  Levers 

(b)  Yoke 

(c)  Release  (throw  out)  Bearing 

(d)  Collar 

(2)  Hydraulic 

(a)  Master  Cylinder 

(b)  Slave  Cylinder 

(c)  Hydraulic  lines 

(d)  Also  includes  (b),  (c),  and  (d)  under  mechanical 

Figure  II.  Objective  teaching  points,  and  MVE 
for  a PLATO  lesson  on  dutches. 
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MVE:  2.  Operation  of  Clutch  Components 


INSTRUCTIONS:  Match  the  lettered  statement  or  phrase  on  the  right  with  the  numbered  statement  or 


phrase  on  the  left. 
_Driving  Member 
Conventional  Clutch  Principle 
Driven  Member 


1. . 

2.  . 

3.  

4.  Hydraulic  linkage  Component 


5. 

6. 

7. 

8. 


Part  of  Mechanical  linkage  in  direct  contact 
with  lelease  bearing 

Strengthens  Disc 

_Performs  the  same  function  as  the  large  coil  in  Single 
Spring  Clutch 

_Does  not  rotate  with  the  flywheel  when  clutch  is  engaged. 


a.  Solid  Friction 

b.  Throw  out  Bearing 

c.  Master  Cylinder 

d.  Asbestos  Fibers 

f.  Helical  Springs 

g.  Yoke 

h.  Friction  Plate 

i.  Copper  Wires 


Figure  II  (Continued) 


Mr.  Arthur  then  collected  available  information  to  familiarize  himself  with  the  subject  area.  He  asked 
instructors  about  the  difficulties  they  had  teaching  airmen  about  clutches,  any  techniques  they  used,  what 
training  devices  were  in  current  use,  et  cetera.  He  examined  career  field  training  literature,  and,  being  a 
thorough  person,  he  looked  through  library  and  personal  materials.  Having  gained  an  awareness  of  the 
efforts  involved  for  the  lesson  lie  got  together  with  the  other  members  to  discuss  ideas,  but  mainly  to  verify 
the  relationship  between  objective,  MVE  and  teaching  points.  This  operation  was  critically  important  for  it 
fixed  the  scope  and  d<*pth  of  training.  Having  accomplished  the  research  step,  Mr.  Arthur  was  confident 
that  he  knew  what  the  student  was  expected  to  leam  about  clutches. 

Shielding  himself  from  distractions,  Mr.  Arthur  mentally  mapped  out  the  lesson.  He  considered 
instructional  strategies,  types  of  interactive  routines,  use  of  graphics,  locations  for  microfiche  slides,  and 
potential  uses  of  the  touch-sensitive  panel  that  detects  screen  locations.  He  knew  it  was  best  to  plan  for 
features  like  microfiche  far  in  advance  since  procurement  was  often  a lengthy  process. 

After  mulling  over  everything,  Arthur  drew  out  a programming  plan  (see  Figure  12)  for  the  lesson. 
Arthur  was  not  content,  for  he  would  have  liked  to  try  a more  innovative  approach  which  he  saw  another 
author  use  before  the  PLATO  and  ISD  projects  were  merged.  Nevertheless,  he  knew  time  was  short,  and  the 
lean  programming  philosophy  pointed  toward  testing  simple  designs  first.  Sgt.  PIP,  the  instructional 
programmer,  reviewed  the  plan  and  noticed  three  things  he  did  not  like  about  lesson  strategy.  First  of  all, 
the  step  size  was  too  large;  i.e.,  the  interaction  was  massed  together  into  two  lumps  after  the  presentation 
of  substantial  amounts  of  material.  Secondly,  Pip  felt  uneasy  since  Arthur  demanded  total  recall:  the 
student  could  not  view  the  information  presented  while  doing  a matching  exercise  called  for  in  the  lesson. 
Finally,  this  matching  exercise  just  was  not  Pip’s  way  of  getting  the  student  involved  with  the  principles  of 
operation  of  clutches.  Far  too  often  he  had  seen  the  authors  use  interactive  technqiues  that  required  simple 
recognition  of  a statement.  Feeling  recalcitrant,  Arthur  rejected  Pip’s  negative  comments  as  opinions. 
Arthur  stated  that  PLATO  was  a new  media  and  the  approaches  used  should  not  be  governed  by  past 
prejudices  derived  from  experiences  with  other  media.  Rp  felt  like  retaliating  by  mention  ng  that  most 
instructional  programming  principles  are  independent  of  media.  But  he  declined,  partly  because  Arthur 
could  be  right -the  machine’s  arcane  functioning  and  capabilities  befuddled  him-but  mostly  because  he 
remembered  his  own  recalcitrance  when  he  first  got  into  the  instructional  programming  career  field.  Rp 
knew  that  student  tryouts,  revisions,  and  the  multiple  attempts  at  lesson  validation  would  soon  modify 
Arthur’s  opinion.  Considering  his  feelings  and  constraints,  he  decided  that  the  innovation  associated  with 
this  project  must  be  tempered  with  moderation;  Arthur  would  have  to  yield. 
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UNIT  1 16  - CLUTCHES,  OBJECTIVES  72.  CHA  1 16 


SHOW  A LABELED  DIAGRAM 
OF  THE  CLUTCH  AND 
COMPONENTS 


STUDENT  MATCHES  FIGURES  TO 
THE  NAMES  OF  THE 
COMPONENT  (WITH  FEEDBACK) 


PRESENT  A DESCRIPTION 
OF  THE  BASIC  PRINCIPLES 
OF  OPERATION 


STUDENT  MATCHES  DESCRIPTIONS 
OF  THE  PRINCIPLES  OF 
OPERATION  (WITH  FEEDBACK) 


Figure  12.  Programming  Plan. 


After  compromising  with  Pip,  Arthur  consulted  with  his  team  SME,  Sergeant  Gear.  From  Arthur’s 
research  notes  and  verbal  descriptions,  Gear  and  he  concluded  that  the  proposed  lesson  contained  the 
information  necessary  for  the  student  to  pass  the  MVE  and  that  it  was  technically  accurate.  Arthur  was  glad 
that  this  lesson  planning  step  was  over  and  that  he  could  now  get  to  work  on  the  lesson. 

Normally,  Arthur  would  start  off-line  lesson  preparation  by  anticipating  the  use  of  microfiche,  so  that 
all  the  steps  involved  in  that  process  would  be  complete  by  the  time  the  lesson  was  to  be  tried  out. 
However,  he  decided  to  use  computer  graphics  as  the  exclusive  vehicle  for  component  identification.  Arthur 
listed  the  drawings  he  would  need,  then  went  to  work  on  outlining  or  writing  the  test  for  various  portions 
of  the  lesson.  He  also  wrote  the  type  of  interactions  the  student  would  be  required  to  do,  like 
multiple-choice  questions,  matching,  fill-in-the-blank,  et  cetera.  Guiding  him  throughout  this  period  is  his 
qualitative  assessment  of  the  ability  of  students  to  read,  comprehend,  and  retain  the  information,  and 
operate  the  terminal.  With  the  reading  problems  some  of  the  students  have,  he  wondered  how  they 
graduated  from  grade  school,  let  alone  high  school.  Using  the  keyboard  was  another  constraint;  Arthur 
decided  not  to  use  constructed  response  items  because  he  noticed  that  some  students  disliked  typing  or 
were  very  slow  at  it. 

With  conceptualization  and  some  written  portions  of  the  initial  version  of  the  clutch  lesson  complete, 
Arthur  felt  it  was  time  to  initiate  on-line  coding.  Mr.  Atwright,  the  TUTOR  coding  specialist,  and  Arthur 
decided  that  the  coding  task  could  be  handled  most  expeditiously  if  Atwright  were  to  assist  in  the  creation 
of  certain  clutch  component  diagrams  and  some  of  the  more  complex  coding  routines  necessary  to  drive  the 
master  validation  examination  and  link  the  lesson  with  the  student  routing  and  data  collection  routines.  In 
some  respects  Arthur  wanted  Atwright  to  do  all  the  coding,  for  after  2 years  he  was  satiated  with  the  task 
and  was  not  looking  forward  to  the  plethora  of  difficulties  ahead.  Atwright  thrived  on  computer 
programming,  it  was  his  job  and  what  he  really  cared  about.  He  knew  of  the  tasks  the  other  team  members 
were  expected  to  perform  and  that  was  sufficient  for  his  needs.  When  he  encountered  problems,  they  were 
regarded  as  challenges;  he  got  great  satisfaction  from  his  developed  problem  solving  skills.  Arthur  knew, 
however,  that  in  order  to  maintain  his  level  of  proficiency,  he  would  have  to  continue  coding  regularly. 
This  was  a source  of  aggravation  to  Arthur,  since  he  was  expected  to  be  a subject-matter  expert,  instructor, 
instructional  programmer,  as  well  as  being  qualified  to  code  in  the  TUTOR  language,  yet  his  job  description 
said  he  was  an  instructor.  Arthur  and  the  other  civilian  authors  had  attempted  to  change  their  position 
description  so  it  would  be  commensurate  with  their  actual  activities.  It  became  a crusade  and  a way  of 
releasing  frustrations  accrued  over  the  initial  phases  of  the  project. 

Of  all  tasks  associated  with  developing  an  on-line  lesson,  the  coding  requires  the  most  time  and 
expenditure  of  mental  energy.  Arthur  believes  it  would  be  simpler  to  teach  nuclear  physics  to  monkeys 
than  code  a lesson.  At  least  with  a monkey  you  can  assume  it  knows  something;  you  don’t  have  to  be 
explicitly  precise.  Arthur  used  a certain  subset  of  TUTOR  commands  available  to  him,  often  in  most 
inefficient  ways.  However,  as  long  as  the  terminal  displayed  or  did  what  he  wanted  it  to  do,  he  was 
satisfied.  On  the  other  hand,  Atwright  strived  for  machme  efficiency,  and  wondered  how  much  time  could 
be  saved  if  all  PLATO  authors  strived  for  the  same  efficient  machine  coding.  Arthur  did  not  wonder;  at  the 
end  of  6 hours  of  looking  at  the  plasma  panel,  he  was  ready  to  go  home. 

Together,  Arthur  and  Atwright  spent  68.5  hours  to  code  the  clutch  lesson.  The  60.5  hours  Arthur 
expended  occurred  over  a 3-week  period.  Satisfied  that  the  lesson  was  ready  for  the  pre-tryout  review, 
Arthur  informed  his  team  members  that  they  could  perform  their  editing  functions.  The  instructional 
programming  and  grammatical  edit  by  Pip  went  quickly.  He  had  checked  periodically  with  Arthur  in  the 
lesson  preparation  phase  when  he  wanted  relief  from  the  other  ISD  projects  and  additional  duties  which 
demended  his  time.  Consequently,  his  familiarity  with  the  lesson  and  the  words  Arthur  was  most  likely  to 
misspell  made  the  task  easier.  Pip  did  not  care  for  Arthur’s  reliance  on  computer  graphics.  Pip  hoped  that 
the  real  component  would  be  available  as  an  adjunct  to  the  lesson.  However,  the  initial  group-paced  system 
tryout  would  require  six  or  more  multiple  copies  of  equipment.  Anyway,  one  concern  w«  to  determine  if 
PLATO  could  reduce  equipment  needs. 
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Technical  edit  by  Gear  also  went  quickly,  but  two  technical  inaccuracies  were  found.  These  he  listed 
in  the  lesson  documentation  folder  where  a history  of  the  development  of  the  lesson  was  supposed  to  be 
kept  by  team  members.  At wright  searched  the  lesson  for  coding  bugs  by  imitating  the  behaviors  of  the 
curious  and  mischievious  students.  The  he  reviewed  the  TUTOR  coding  (see  Figure  13)  as  the  last  activity  in 
the  pre-tryout  review. 


1 

unit 

quiz  1 

2 

term 

quiz  1 

3 

calc 

n6<>0 

4 

back 

t9 

5 

do 

dr i vere  ( 1 ) 

6 

pack 

n 1 , caaabdbdad 

7 

at 

1524 

8 

write 

<at , 1 625>UJhat  is  the  component  indicated  below? 

9 

write 

<at , 1 925>a. 

10 

b. 

11 

c. 

12 

d. 

13 

calc 

n 124*0 

14 

entry 

qu  i rea  1 

15 

at 

2521 

16 

erase 

10 

17 

calc 

nl24»nl2  + l 

18 

jump 

nl2*l 1 ,out,x 

19 

join 

nl2-2, f in,csp,goo,ped, f ly, spr , bac, pre, dis, thr , 

20 

calc 

n2>rit  (nl2)  $cls$54 

21 

arrow 

2521 

22 

long 

1 

23 

storea 

n4 

24 

specs 

bunpshi ft 

25 

judge 

n2*n4,ok,no 

26 

j oin 

ansok,yes,bad 

27 

endarrow 

28 

mode 

erase 

29 

join 

nl2-2, f in,csp,goo,ped, fly, spr , bac, pre, dis, thr „ 

30 

mode 

write 

31 

goto 

qu l rea 1 

Figure  13.  TUTOR  code. 
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The  student  tryout  of  the  clutch  lesson  was  the  first  in  which  Arthur  was  an  active  participant.  The 
tryout  step  actually  consisted  of  three  distinct  tryouts  using  one,  then  two,  and  finally  three  students  of 
varying  ability.  Some  lesson  revision  took  place  in  between  tryouts,  but  Arthur  was  reluctant  to  make 
major  changes  based  on  the  information  from  a small  sample  of  students.  He  would  wait  until  lesson 
validation  data  were  obtained  during  the  system  tryout.  He  disliked  the  idea  of  his  lesson  being  put  “on  the 
shelf’  until  then  when  it  could  be  used  by  the  course.  The  ISD  team  had  decided  against  this  since  PLATO 
system  instability  might  alienate  the  course  personnel,  and  the  time  and  manpower  diversion  from 
courseware  development  could  not  be  spared.  Besides,  the  lesson  validation  data  relevance  is  enhanced 
when  the  system  is  functioning  as  a unit,  all  the  new  courseware  interacting. 

The  clutch  lesson  would  validate  when  90%  of  the  students  achieved  the  objective  standard  of  75%. 
Approximately  30  students,  or  three  consecutive  classes,  constitute  the  validation  sample.  If  more  than 
three  students  failed  to  reach  the  standard  even  though  only  two  clases  experienced  the  most  current 
version,  it  was  revised. 

The  clutch  lesson  underwent  several  changes  before  it  was  validated.  After  the  first  two  classes,  it  was 
apparent  that  the  lesson  had  some  deficiencies.  Arthur  tried  to  diagnose  deficiencies  and  determine  a 
remedy.  He  knew  that  he  would  not  have  much  assistance  in  accomplishing  the  task;  by  this  time,  the  team 
approach  was  almost  nonexistent.  Over  one-third  of  the  19  project  personnel  had  left,  and  as  of  them, one 
untrained  author  had  been  acquired  as  a replacement. 

Arthur  accessed  the  data  collection  file  and  selected  options  four  and  seven  for  instructional  unit  305, 
clutches  (Figures  14,  15,  and  16).  He  wanted  to  see  which  students  were  failing  to  meet  the  criterion,  as  well 
as  review  the  item  analysis  on  the  MVE.  After  comparing  this  data  with  that  obtained  from  the  prior 
validation  attempt  and  examining  the  written  comments  the  instructors  of  eacn  class  had  given  to  the  ISD 
team,  Arthur  felt  that  he  knew  why  the  students  were  having  difficulty.  Some  textual  reconstruction  and 
the  addition  of  several  additional  questions  seemed  appropriate.  Several  hours  later  the  adjustments  had 
been  programmed  and  coded,  but  it  would  be  2 weeks  before  the  next  class  would  get  to  the  lesson. 

Ten  months  after  Arthur  initiated  the  development  of  instructional  unit  305,  Clutches,  the  lesson  was 
found  to  be  acceptable  for  student  use  and  underwent  a final  grammatical  edit  and  an  acceptance  check  by 
Special  Purpose  Vehicle  Repairman  course  management.  It  became  a part  of  the  inventory  of  new 
courseware  used  in  the  Special  Purpose  Vehicle  instructional  system.  A portion  of  the  lesson  (without 
animation,  response  judging  and  branching)  follows. 
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For  individual  student  data.  Data  relative  to  the 
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Figure  14.  “CHA  95”  data  retrieval  and  analysis  options. 
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Figure  IS.  Selected  MVE  data. 


igure  16.  Incorrect  response 


Frame  2 


The  clutch  connects  and  disconnects  the  engine 
from  the  manual  transmission.  The  siutch  must 
do  this  smoothly  to  prevent  jerky  starts  and 
shock  to  the  parts  of  the  power  train. 
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Frame  3 


Tbi;s  is  a cross  sectional  view  of  the  entire  CLUTCH 
ASSEMBLY.  Next  I will  give  you  tne  names 
of  the  individual  parts 


Frame  4 


Recognize  this?  We  called  it  part  of  the  engine 

before right  now  think  of  it  as  part  of  the 

clutch  assembly. 

It's  called  the  FLYWHEEL. 


Frame  5 


Now  this  is  new... Bolted  right  to  the  flywheel 
is  the  PRESSURE  PLATE  BACKING 


Frame  6 


Frame  7 


The  inner  surfaces  of  the  FLYWHEEL  and 
PRESSURE  PLATE  (see  the  arrows)  are  machined 
and  ground  to  a smooth  finish.  They  are  made 
of  cast  iron  which  contains  enough  graphite  to 
provide  some  lubrication. 


Frame  8 


<> 

<> 


The  PRESSURE  PLATE  is  directly  connected  to  the 
PRESSURE  PLATE  BACKING.  (This  connection  is  not 
shown  in  the  diagram  ) Remember  that  they  both 
turn  together! 
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Frame  9 


The  three  parts  you  see  here  are  all  knouirv  as 
"drivING  members".  This  is  because  they  all 
turn  with  the  engine. 

The  next  part  you  will  see  is  known  as  the 
“drivEN  member".  Because  it  is  not  directly 
connected  to  the  engine. 


Frame  10 


This  is  the  CLUTCH  CISC  (or  FRICTION  PLATE? . 
Notice  that  it  fits  between  two  of  the  driving 
members,  the  flywheel  and  the  pressure  plate. 

clutch  disc  *-*  drivEN  member 
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I 


ENGINE 


L 


This  is  the  CLUTCH  DISC  (or  FRICTION  PLAT©  . 
Notice  that  it  fits  between  two  of  the  driving 
members,  the  flywheel  and  the  pressure  plate. 


clutch  disc  «-►  drivEN  member 

The  clutch  disc  is  connected  to  the  input  shaft 
of  the  transmission,  by  splines!!  This  allows 
the  disc  to  slide  ba^k  and  forth  on  the  shaft! 

v 


Frame  11 


The  Pressure  Plate  is: 

a.  a driving  member 

b.  a driven  member 

c.  an  operating  member 

d.  linkage  component 
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Frame  12 


The  Pressure  Plate  Backing  is: 

a.  a driving  member 

b.  a driven  member 

c.  an  operating  member 

d.  linkage  component 


Frame  13 


The  Clutch  Disc  is: 

a.  a driving  member 

b.  a driven  member 

c.  an  operating  member 

d.  linkage  component 
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ML.  i i 


Frame  IU 


Here  are  5 varieties  of  clutch  discs: 

1.  DRV  This  is  the  conventional  clutch  disc.  It  is 

made  of  spring  steel  with  facings  riveted 
to  each  side.  The  facings  are  made  of  molded 
cotton  and  asbestos  fibers.  Often  copper  wires 
are  woven  into  the  material  for  strength. 


2.  WET 


Oil  is  supplied  to  the  disc  surface  to  provide 
for  cushioning  between  the  surface  of  the  disc 
and  drivING  members. 


3.SINGLE  Has  only  one  clutch  disc 

4.  MULTIPLE  Has  2 or  more  discs.  These  are  used  on 

vehicles  with  heavy  loads. 

5.  DISHED  DISC 

Disc  is  dished  so  that  the  inner  and  outer 
surfaces  of  the  facing  make  contact  gradually  as 
spring  pressure  is  increased  and  the  disc 
flattens  out. 


i*er  miABi[  copy 


Frame  37 


What  is  the  component  indicated  below? 


_r 

RJ 


a . 1 1 nkage 

b.  pressure  plate 
multiple  disc 


c. 


d.  release  bearing 


> 


Frame  38 


You  got  7 out  o f a poss i b 1 e 1 0 quest l ons 
correct  on  the  first  try. 
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KV  MVABVt 

'}  Opera. 1 1 ng  pr  1 nc  i p 1 es : 


Frame  39 


The  operation  of  the  clutch  is  as  follows: 

When  the  clutch  is  fully  engaged  the  driven  disk 
is  firmly  clamped  between  the  flywheel  and  the 
pressure  plate  by  the  pressure  of  the  springs. 

When  the  driver  disengages  the  clutch  by  depressing 
the  pedal,  the  release  yoke  or  fork  is  moved  on  its 
pivot,  and  pressure  is  applied  to  the  release  bearing 
sleeve,  or  collar,  containing  the  release  bearing. 

The  rotating  race  of  the  release  bearing  presses 
against  the  clutch  release  levers  and  moves  them  on 
their  pivot  pins.  The  outer  ends  of  the  release 
levers,  being  fastened  to  the  cover,  move  the 
pressure  plate  to  the  rear,  compressing  the  clutch 
springs  and  allowing  the  driving  members  to  rotate 
independently  of  the  driven  member. 


Frame  40 


Operat  i ng  pr  i nc  i p 1 es  (cont  i nued) 

Mil  parts  of  the  clutch,  except  the  release 
bearing  and  collar,  rotate  with  the  flywheel  when 
the  clutch  is  engaged. 

When  the  clutch  is  disengaged,  the  release  bearing 
rotates  with  the  flywheel,  but  the  driven  disk,  and 
the  clutch  shaft  come  to  rest. 
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Frame  42 


- ' 


- I 


1 C€  the  disengaged  position  snown  aboye 

that  the  pedal  is  depressed! 


Frame  43 


Now  the  clutch  is  shown  in  the  engaged  poet iorv 

fts  the  clutch  is  engaged  slowly  (in  normal  operation) 
the  vehicle  will  start  to  creep  even  though  there 
is  a great  deal  of  slippage.  The  engine  cannot 
transmit  its  maximum  power  to  the  transmission 
until  maximum  friction  is  obtained  between  the 
clutch  members.  That  is,  when  all  slipping  has 
stopped  and  there  is  a direct  connection  between 
the  driving  and  driven  parts. 


I mine  4 1 


^°t ice  -hat  in  the  engaged  position  shown  above 
that  the  pedal  is  released! 


Frame  58 
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Component 
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d 1 rec':  contact  with  release 
bearing  collar 


6.  Strengthens  Disc 

7.  Performs  the  same  function  as  the 

1 arge  coil  l n S l ng ) e Spr i ng  Cl ut ch 

3.  Does  not  rotate  with  the  flywheel 

when  clutch  is  engaged 


era  WAIlAB'i  COPY 


l;r;imc  5‘) 


Match  the  lettered  statement  or  phrase  on  the  right 
with  the  numbered  statement  or  phrase  on  the  left. 

a . So 1 i d F r i ct 1 on 

b.  Throw  out  Bearing 

1 e Driving  Member  c.  Master  Cylinder 

d.  Asbestos  Fibers 

2.  a Conventional  Clutch  Principle  e.  Pressure  Plate 

f . Hell ca 1 spr l ngs 

3.  h Driven  Member  g.  Yoke 

h . Fr  i ct  i on  F1  late 

H.  c Hydraulic  Linkage  Component  i.  Copper  Wires 

5.  g Part  of  Mechanical  Linkage  in 

d i r eco  cent  act  w i t h re  1 ease 
bearing  collar 

6.  l Strengthens  Disc 

7.  f Performs  the  same  function  as  the 

1 arge  co l 1 in  Si ng 1 e Spr i ng  C 1 ut ch 

8.  b Does  not  rotate  with  the  flywheel 

when  clutch  is  encaged 


Press  -NEXT-  to  change  or  correct 

answers.  Press  -DATA-  to  have  this  test  graded. 
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Frame  C>0 


a.  Solid  Friction 

b.  Throw  out  Bear  in: 

c.  Master  Cylinder 

d.  Asbestos  Fibers 

e.  Pressure  Plate 

f . Hell ca 1 spr 1 ngs 

g.  Yoke 

h . Fr i ct ion  PI at e 
1 . Copper  10 1 res 


ponent 


c Hydra.u  lie  L i nk 


Part  of  Mechanical  Linkage 
d i rec  cent  act  with  re  1 ease 
bearing  collar 


f Performs  the  same  function  as  the 
1 arge  co l 1 in  Single  Spr i ng  Cl utch 


Does  not  rotate  with  the  flywheel 
when  clutch  is  engaged 


Press  -NEXT-  to  change  or  correct 
answers.  Press  -DATA-  to  have  th 


Frame  61 


beh i nd 
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